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ABSTRACT 


An  investigation  is  made  of  the  Navy  Exploratory  Develop- 
ment Program;  its  newly  revamped  management  system,  its 
organization  and  underlying  rationale.  The  investigation 
then  addresses  the  processes  of  program  selection,  review  and 
evaluation  as  applied  to  technology  base  programs,  followed 
by  an  overview  of  a methodology  called  "Evaluation  Research" 
as  it  applies  to  goal  oriented  programs.  The  paper  is  dir- 
ected toward  determining  the  effectiveness  of  technology  base 
programs,  with  a major  emphasis  on  the  changing  structure  of 
the  Exploratory  Development  Program  within  the  Department  of 
Defense. 
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I. 


INTRODUCTION 


There  has  never  been  a period  in  history  in  which  the 
determination  of  the  worth  of  programs  within  the  Department 
of  Defense  (DoD)  structural  framework  has  been  so  critically 
viewed.  Adding  to  this  observation  is  the  discernible  fact 
that  relative  budgets  within  the  Research  and  Development 
(R&D)  Program  are  steadily  decreasing. 

This  concern  encompasses  total  system  acquisition,  oper- 
ations and  maintenance,  hardware  and  software  development, 
concept  feasibility,  pure  research  and  all  other  programs 
intermediate.  Further,  the  concern  is  broad-based  to  the 
extent  that  it  has  become  a prime  distress  not  only  to  the 
government  sector  of  DoD,  but  also  to  the  supporting  indus- 
trial counterparts. 

Owing  to  the  desire  then  to  better  justify  on-going  pro- 
grams, to  promote  innovation  and  concentration  on  highest 
leverage  opportunities,  to  expedite  the  transfer  of  technolgy 
and  projects  to  advanced  development  (category  6.3)  and  use 
of  new  technical  capabilities  developed  in  research  (category 
6.1)  and  to  demonstrate  relevance  and  cost  effectiveness, 
the  Research  and  Development  program  in  total  is  currently 
undergoing  a modification  of  the  existing  management  planning 
system,  particularly  the  Exploratory  Development  Program  (EDP) . 

The  objective  of  this  paper  then  is  really  twofold: 

(1)  to  investigate  the  changing  structure,  organization  and 
underlying  rationale  of  the  Exploratory  Development  Program 
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in  the  Navy  and  (2)  to  investigate  the  processes  of  program 
selection,  review  and  evaluation  as  applied  to  technology 
programs  within  the  EDP . Included  will  be  a brief  on  the 
development  of  the  EDP,  a treatise  on  why  it  is  a necessary 
and  contributing  element  of  R&D  and  also  a discussion  of  what 
the  future  might  hold  for  determining  the  "worth"  of  technol- 
ogy type  programs. 

Because  of  the  significance  placed  on  an  understanding 
of  the  operations  of  the  EDP  as  a necessary  and  crucial  pre- 
requisite to  developing  a meaningful  evaluation  methodology, 
the  majority  of  this  paper  is  focused  on  identifying  the 
foundations  and  workings  of  the  newly  revamped  Exploratory 
Development  Program.  This  understanding  is  essential  before 
any  succinct  attempt  can  be  made  to  alleviate  the  discrepan- 
cies in  the  review  and  evaluation  techniques,  existing  today. 
Hopefully,  a closer  interpretation  of  the  EDP  will  facilitate 
a more  viable  methodology  for  determining  the  Return-on-Invest- 
ment  (ROI)  of  future  technology  programs  within  DoD. 
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II.  BACKGROUND 


A.  THE  TECHNOLOGY  BASE 

Somewhere  between  basic  Naval  research  on  one  end  of  the 
Research  and  Development  (R&D)  spectrum  and  full  scale  devel- 
opment on  the  other,  fall  a great  many  projects  which  come 
under  the  heading  of  applied  technology,  or  technology  base 
programs.  The  technology  base  is  the  total  reservoir  of 
organized  knowledge  from  directly  and  indirectly  sponsored 
basic  research  into  physical  and  social  phenomena  and  the 
feasibility  of  new  processes,  techniques  and  components  for 
using  them  (Ref.  1) . Its  end  result  is  new  organized  knowl- 
edge. The  base  also  retains  fallout  information  and  contri- 
butions left  over  from  exploration  and  development  of  specific 
systems.  The  creation  and  exploration  of  candidate  systems, 
in  turn,  is  shaped  by  the  information  available  from  the  tech- 
nology base. 

One  of  the  largest  and  perhaps  most  dynamic  businesses  in 
the  United  States  today  is  that  dealing  with  the  acquisition 
of  major  weapon  systems  for  the  Department  of  Defense.  Due  to 
the  size  and  complexity  of  the  federal  government  and  its 
associated  agencies,  management  of  the  affairs  of  this  organi- 
zation requires  a tremendous  amount  of  activity  and  coordination. 

This  business  entity,  not  unlike  any  other,  maintains  a 
growth  posture  made  possible  by  both  internal  and  external 
influences.  These  influences  are  visible  through  the  technol- 
ogy development  programs  supported  by  the  organization.  Thus, 
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a most  important  faction  of  the  R&D  community  within  the 
Department  of  Defense  (DoD)  is  that  responsible  for  developing 
and  maintaining  a strong  technology  base. 

The  technology  base  is  supported  by  both  Government  and 
private  activities.  Private  groups  (such  as  industrial  com- 
panies) and  nonprofit  organizations  (such  as  Federal  contract 
research  centers  and  universities)  both  contribute  to  the 
technology  base  and  recommend  technology  base  programs,  just 
as  do  the  federal  laboratories  and  other  supporting  agencies. 

Within  DoD,  the  defense  laboratories  are  charged  with  main- 
taining the  technology  base  by  sponsoring  activities  in  support 
of  potential  systems.  Laboratory  research  objectives  are 
planned  to  generate  better  technological  tools  five  to  twenty 
years  in  the  future.  There  does  indeed  exist  a requirement 
for  the  technology  to  be  related  to  mission/operational  needs. 

Due  to  the  seemingly  nebulous  nature  of  these  technology 
oriented  programs,  a great  deal  of  scrutiny  has  been,  and  con- 
tinues to  be,  given  them,  with  primary  emphasis  of  the  concern 
aimed  at  their  resultant  value.  Because  of  the  presence  of 
intangible  outputs  in  the  sense  of  quantification,  coupled 
with  an  evidence  of  decreasing  relative  budgets,  technology 
administrators  at  all  levels  within  the  command  have  been,  and 
continue  to  be,  faced  with  the  dilemma  of  determining  which 
programs  should  receive  their  attention.  Their  problem  is 
essentially  twofold:  (1)  How  are  they  to  determine  which  of 
the  many  proposed  (alternative)  programs  should  be  selected 
for  support,  and  (2)  how  are  they  to  evaluate  the  effectiveness 
of  their  programs? 
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It  is  very  difficult  to  know  how  much  should  be  spent  for 
technology  base  activities  because  payoffs  are  virtually 
unpredictable  and  only  rarely  are  they  realized  at  some  future 
date  in  a directly  traceable  manner  (Ref.  1) . Operational 
needs  can  be  met  most  effectively  at  the  lowest  possible  cost 
by  innovative  products  built  on  new  technology  being  created 
for  its  own  sake  (pure  technology)  as  well  as  that  which 
remains  as  fallout  from  other  system  efforts  (applied 
technology) . 

A significant  problem  in  defense  has  been  to  control  the 
relationship  between  technology  base  activities  and  system 
acquisition  programs.  Technology  base  projects  have  gone  on 
to  develop  more  than  just  the  raw  elements  and  material  to 
meet  as  yet  unexpressed  needs.  Traditionally  they  have  devel- 
oped subsystems  and  system  concepts  independently  of  specific 
needs.  Once  developed,  subsystems  have  sought  out  homes  in 
new  system  candidates  and  systems  in  turn  have  searched  for  a 
need  they  could  satisfy.  Further,  these  subsystems  have  been 
old  technologies  extended  to  improve  the  performance  of  old 
kinds  of  products. 

There  is  a common  misconception  that  defense  systems  have 
suffered  because  they  have  embodied  too  much  "new  technology" 
and,  as  a result,  have  become  too  complex,  sophisticated  and 
expensive  (Ref.  1) . The  explanation  for  this  is  that  what  is 
"new"  about  "new  technology"  is  the  higher  level  of  perform- 
ance demanded  from  the  old  technical  approaches.  In  fact. 
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subjective  ratings  have  indicated  that  systems,  while  becoming 
more  complicated  and  expensive,  have  become  less  technologi- 
cally advanced. 

Applying  new  technology  to  meet  more  stringent  needs  can, 
in  fact,  result  in  simpler,  less  costly  systems.  As  a common 
example,  the  transistor  development  has  markedly  reduced  the 
cost  and  size  of  electronic  systems  and  improved  their  reli- 
ability, maintainability  and  other  cost  of  ownership  factors. 

The  need  for  an  active  and  viable  technology  base  is  thus 
extremely  important.  The  larger  the  span  of  technical  and 
scientific  activity  sponsored  by  the  Government,  the  more 
extensive  will  be  the  optional  technological  choices  avail- 
able to  the  system  designer.  It  is  imperative  that  cost- 
benefits  be  maximized  in  providing  the  technology  base  for 
new  systems.  This  then  can  only  be  realized  when  the  base 
program  is  driven  from  operational  needs  and  mission  require- 
ments. This,  in  turn,  requires  a most  judicial  and  expedient 
methodology  for  determining  the  criteria  for  the  selection 
and  evaluation  and  the  effectiveness  of  the  technology  base 
programs  in  the  R&D  environment. 

Just  as  critical  as  a well  defined  and  operable  methodology 
for  determining  selection  and  evaluation  criteria,  is  the 
understanding  of  the  actual  program,  or  system,  which  is  to 
be  operated  on.  No  attempt  could  ever  be  made  to  determine  a 
program's  worth,  or  return  on  investment  (ROI),  without  a prior 
and  thorough  understanding  of  the  functions  and  underlying 
philosophies  of  that  program.  It  is  crucial  that  a sound 
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foundation  be  established  from  which  goals  and  strategies 
can  be  derived  rather  than  the  traditional  dollar  value  of 
output. 


3.  THE  EXPLORATORY  DEVELOPMENT  PROGRAM  AS  THE  CRITICAL  LINK 

While  many  definitions  of  exploratory  development  exist, 
there  are  several  functions  that  should  be  fulfilled  by  this 
category  of  Research  and  Development.  These  might  be  grouped 
into  the  following  areas; 

1.  The  translation  and  application  of  research  results 
to  practical  use,  including  the  formulation  of  new 
technology  areas. 

2.  Feasibility  demonstrations  of  component  and  system 
hardware  concepts. 

3.  Applications  of  new  technology  to  existing  components 
or  systems. 

The  nature  of  the  preceding  functions  implies  that  both  the 
inputs  and  outputs  of  exploratory  development  are  quite 
varied,  and  can  be  thought  of  as  a continuum.  Maximum  benefit 
in  an  Exploratory  Development  organization  would  appear  to 
accrue  from  having  the  continuum  functions  in  close  proximity. 

The  purpose  of  the  Exploratory  Development  program  as 
delineated  by  NAVMATINST  3910. 7B  of  3 October  1974  is  as 
follows: 


The  purpose  of  the  Exploratory  Development  program 
is  to  develop  the  technological  wherewithall  to  solve 
specific  Navy  and  Marine  Corps  problems,  and  to  identify 
and  exploit  new  technological  opportunities  which  may 
stimulate  new  operational  concepts  within  the  operating 
forces.  It  is  the  policy  of  the  Chief  of  Naval  Develop- 
ment that  the  program  include  concept  formulation  effort 
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in  the  form  of  analytical  and  experimental  effort  to 
help  identify  problems,  determine  alternative  solu- 
tions thereto  in  terms  of  prospective  systems,  sub- 
subsystems and  techniques,  and  demonstrate  the  technical 
feasibility  of  those  solutions  to  a degree  which 
warrants  their  consideration  for  support  under  Advanced 
Development.  Selectively,  where  a system  or  subsystem 
is  relatively  small  and  of  low  technical  risk,  the 
program  may  support  effort  aimed  at  acquiring  informa- 
tion prerequisite  to  commencement  of  Engineering 
Development.  Such  effort,  however,  should  stop  short 
of  engineering  refinements  characteristic  of  proto- 
typing development  for  procurement,  inventory  or  opera- 
tional deployment.  While  the  program  may  also  encompass 
analytical  and  experimental  work  on  the  technologies 
directly  related  to  materials,  components,  processes, 
techniques , and  individual  equipments  of  Navy  and 
Marine  Corps  interest,  broad-based  investigations  in 
physical,  biological,  environmental,  medical,  behavioral, 
social  and  engineering  sciences  characteristics  of  the 
Naval  Research  program  are  not  permitted.  Effort  of 
an  exploratory  nature,  which  has  a primary  purpose  of 
direct  support  of  specific  Advanced,  Engineering  or 
Operational  Systems  Development  projects  should  be 
funded  and  accomplished  as  an  integral  part  of  those 
projects,  subject  to  close  coordination  with  related 
effort  in  the  Exploratory  Development  program. 

The  operation  of  this  system  has  several  characteristics 

that  are  significant  in  the  management  of  applied  technology 

(category  6.2)  programs  (Ref.  2). 

• Very  broad  technological  base  requiring  judgements  and 
technical  expertise  in  many  fields  at  all  levels  of  control. 

• Many  layers  of  control — DDR&E , ASN  (R&D) , CND,  SYSCOMs , 
Offices,  DLP,  DNT,  laboratories — and  necessity  for  coordina- 
tion of  tasks  with  related  programs  as  well  as  with  pure 
research  (category  6.1)  and  advanced  development  (category 
6.3)  . 


• Coordination  necessary  between  services  and  with  the 
industrial  complex. 

• Results  of  program  efforts  are  seldom  visible  or  recog- 
nized in  the  system  that  reaches  operational  use. 

• Categorization  of  tasks  is  not  easily  relatable  to  naval 
responsibilities,  missions,  operational  concepts,  or  systems. 
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• CNO's  guidance  on  Naval  needs  is  not  directly  trans- 
latable into  R&D  requirements.  CNM  does  not  interpret  oper- 
ational needs  for  SYSCOMs.  Each  SYSCOM  (and  suborganization 
within  SYSCOM)  employs  a different  subjective  means  in 
defining  R&D  goals.  There  is  no  priority  assigned  to  indi- 
vidual tasks. 

• The  system  creates  a large  number  of  low  level  funded 
tasks  (approximately  650  Task  Area  Plans  (TAPs)  (DD1634) 
yearly) . 

• Approved  TAPs  can  be  underfunded  or  not  funded  at  the 
discretion  of  SYSCOMs  or  laboratories  without  reference  to 
CND  or  DNT. 

• Reductions  in  funding  are  frequently  made  by  propor- 
tionate cuts  which  results  in  perpetuation  of  sub-critical 
programs  and  reduced  effort  on  critical  programs. 

• Excessive  effort  is  devoted  to  making  minor  changes  in 
funding  levels  with  resulting  turbulence. 

• Inordinate  effort  is  devoted  to  management,  funding 
changes  and  justifications  of  programs  which  takes  technical 
personnel  away  from  productive  tasks. 

• The  system  fosters  "micro-management."  Program  reviews 
frequently  do  not  result  in  "make-a-dif f erence"  changes. 

• Protection  of  in-house  effort  emphasized  as  contrasted 
to  contractor  effort. 

• Delay  in  exploiting  available  technology. 

• Overemphasis  on  evolutionary  and  fix-it  projects. 

• Inadequate  scientific  and  technical  advice  in  special- 
ized fields  involving  programs  of  several  organizations. 

• Critical  problems  exist  related  to  coupling  the  6.1, 

6.2,  and  6.3  programs  to  initiate  an  Engineering  Development 
(6.4)  program. 

It  thus  becomes  evident  that  a crucial  prerequisite  to 
any  understanding  of  the  methodologies  of  Exploratory  Develop- 
ment program  selection  and  evaluation  is  a thorough  under- 
standing of  the  program  itself.  For  this  reason,  and  owing 
to  the  fact  that  EDP  is  at  this  time  undergoing  major  organi- 
zational and  structural  revision,  a significant  amount  of 
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attention  is  shown  this  subject  in  this  thesis.  The  EDP  is 
indeed  the  "critical  link"  in  terms  of  developing  major  weapon 
systems  within  the  Department  of  Defense,  and  understanding 
the  functions  and  operations  of  it  are  also  critical  in  terms 
of  measuring  program  worth.  The  changes  in  the  program  have 
been  initiated  in  an  effort  to  alleviate  the  negative  aspects 
of  the  previously  mentioned  characteristics  of  the  program 
operation.  It  is  felt  that  only  through  an  understanding  of 
the  function/operational  aspects  of  the  EDP  can  the  concept 
of  Retum-on-Investment  be  addressed,  since  this  must  ulti- 
mately be  linked  to  the  organizations'  goals  and  strategies 
rather  than  dollar  value  of  output. 
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III.  EXPLORATORY  DEVELOPMENT  PROGRAM 


A.  A RATIONALE  FOR  A POLICY  PLATFORM 

Because  of  the  tremendous  size  and  complexity  of  the 
Federal  Government  and  its  many  associated  agencies,  manage- 
ment of  the  affairs  of  this  organization  constitutes  a very 
significant  amount  of  activity  and  coordination.  Accordingly, 
a great  deal  of  scrutiny  is  given  the  policy  platforms  and 
expenditure  characteristics  within  sub-organizations  of  this 
complex. 

This  section  will  investigate  the  Exploratory  Development 
Program  within  the  Department  of  the  Navy,  examine  the  policy 
of  the  EDP  and  the  rationale  behind  that  policy,  examine 
several  influences  which  are  evidenced  by  the  technology  pro- 
gram, and  provide  a personal  critique  of  that  policy  plat- 
form currently  existing  for  the  EDP  from  the  standpoint  of  a 
public  procurement. 

It  is  felt  that  a comfortable  understanding  of  the  EDP 
policy  is  a valuable  and  key  element  which  allows  a better 
comprehension  of  the  significance  of  the  EDP  structural 
change,  thus  leading  toward  a discussion  of  the  present  pro- 
gram organization  and  characteristics.  The  approach  then 
will  be  to  first  address  the  broader  issue  of  external  policy 
influences  and  then  focus  on  particular  program  policies  as 
delineated  by  the  Assistant  Secretary  of  the  Navy  for  Research 
and  Development  (ASN  (R&D) ) . 
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1.  The  EDP  as  a Public  Program 

If  any  government  program  could  be  labeled  "public," 
the  Exploratory  Development  Program  must  certainly  be  con- 
sidered as  a leading  contender.  The  business  of  this  program 
is  certainly  conducted  for  the  general  welfare  of  the  people 
of  the  United  States  through  the  guise  of  building  and  main- 
taining a strong  and  viable  defense  posture.  Clearly  then, 
the  output  of  this  entity  is  to  be  labeled  a public  good, 
since  it  is  consumed  by  all  members  of  the  community  of  the 
United  States.  The  control  of  expenditures  is  governed  by 
the  internal  processes  within  the  Department  of  Defense  (DoD) , 
since  the  costs  of  production  cannot  be  maintained  by  competi- 
tive pricing. 

Because  of  the  nature  of  the  product  in  the  Explora- 
tory Development  Program,  the  citizens  do  not  actively  par- 
take in  setting  policy  or  in  providing  an  oversight  function. 
This  task  is  left  to  entrusted  officials  within  DoD,  as  well 
as  the  mix  of  activities  performed.  The  financing  aspect  of 
the  EDP  comes  under  close  scrutiny  through  the  auspices  of 
both  Congress  and  the  Office  of  Management  and  Budget  (OMB) 
in  the  Executive  Branch  of  government.  Programs  on-going 
must  continually  be  justified  and  reviewed  for  content,  return, 
applicability  and  need. 

There  is  a facet  of  the  EDP  which  presents  itself  as 
an  externality  to  the  private  sector.  This  is  the  area  which 
is  embodied  by  the  notion  of  "technology  transfer."  This 
term  is  used  to  imply  that  technology,  as  the  output  from  a 


research  effort,  can  be  utilized  productively  in  many  loca- 
tions and  organizations  over  and  beyond  its  initial  specific 
mission  (Ref.  3) . The  process  of  technology  transfer  in  the 
EDP  has  traditionally  been  one  of  primarily  outflux  from  DoD. 

The  working  policy  and  structural  framework  of  the 
EDP  is  built  to  closely  resemble  a standard  "rational  model" 
organization  (Ref.  4) . This  will  become  quite  evident  later 
when  the  working  policy /critique  is  presented.  However, 
implementation  of  this  policy  is  probably  more  closely  repre- 
sentative of  the  classical  "bureaucratic  model,"  with  only 
a moderate  trend  toward  a politically  contrived  system  (Ref. 

5)  . 

It  should  be  submitted  that  no  "Operational  Code" 
exists  which  might  be  any  more  distinctive  than  the  academic 
models  just  mentioned.  This  hypothesis  is  supported  by 
noticing  the  well  defined  organizational  concepts  and  general 
propositions  under  which  the  EDP  functions;  high  degree  of 
standardized  procedures,  uncertainty  avoidance,  centralized 
policy  formulation,  decisions  made  by  leaders,  presence  of 
parochial  priorities,  actions  are  organizationalized,  change 
is  definitely  incremental,  long  range  planning  is  prevalent, 
goals  and  tradeoffs  are  of  primary  concern,  decisions  are 
based  on  administrative  feasibility,  change  is  directed  from 
higher  authority  and  there  is  limited  flexibility  in  the  line 
functions  of  the  organization.  Additionally,  one  can  very 
obviously  witness  well  defined  rules  of  the  game,  actions 
which  come  about  as  a result  of  political  inducements,  definite 
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separation  and  identification  of  chiefs  and  indians  and  much 
mis-communication,  all  of  which  point  toward  a strong  influ- 
ence of  the  political  model  characteristics  (Ref.  4) . 

Having  provided  an  overview  of  the  structure  of  the 
EDP  and  its  operational  characteristics  and  public  attributes, 
attention  now  will  focus  more  closely  on  the  particular  policy 
of  the  program  on  which  its  platform  is  built.  Any  discus- 
sion of  the  Congressional  and  Presidential  influences  on 
the  EDP  has  been  negated  because  it  is  not  felt  to  be  a pro- 
gram, primarily  because  of  its  size  and  technical  complexity, 
which  lends  itself  to  the  normal  control  imposed  by  those 
forces.  All  interactions  which  do  exist  are  enacted  at  the 
ASN  ( R&D)  level,  the  results  of  which  follow. 

2.  EDP  Rationale 

A review  of  the  Navy's  Exploratory  Development  Pro- 
gram by  the  Assistant  Secretary  of  the  Navy  (R&D)  led  to  the 
recent  concern  to  re-establish  the  policies  which  are  intended 
to  govern  that  section  of  Research  and  Development.  As  a 
result,  the  ASN  (R&D)  has  specified  the  policies,  performance 
objectives,  and  responsibilities  concerned  with  the  overall 
management  of  the  Department  of  the  Navy  Exploratory  Develop- 
ment Program  (Ref.  6) . 

It  is  the  belief  of  the  ASN  (R&D)  that  improvements 
are  possible  in  technical  strategy,  responsiveness  to  innova- 
tion, transferring  results  into  useful  applications,  visi- 
bility to  management,  and  demonstration  of  the  rationale  and 
cost  effectiveness  of  the  Exploratory  Development  Program  and 
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its  elements.  The  underlying  rationale  of  the  Exploratory 
Development  Program  is  derived  from  the  fact  that  this 
activity,  like  all  others,  must  compete  for  its  resources. 
This  implies  that  the  programs  be  continually  justified  in 
meaningful  and  convincing  terms . 

According  to  the  ASN  (R&D) , to  assure  that  the  long 
term  benefits  are  clear.  Exploratory  Development  must  demon- 
strate how  it  (Ref.  6) : 

a.  Provides  technical  opportunities  which  stimulate 
the  development  of  substantially  improved  and/ 
or  totally  new  operational  capabilities. 

b.  Provides  solutions  to  problems  arising  in  the 
developmental  process. 

c.  Provides  opportunities  to  regain  operational 
superiority  or  to  exploit  and  enhance  an 
existing  operational  superiority. 

d.  Provides  for  the  transformation  of  new  research 
discoveries  into  useful  applications,  and  for 
the  innovative  extension  of  existing  technologies 
to  significantly  more  advanced  applications. 

3.  Policy  Formulation 

Using  as  a foundation  the  principles  implicit  in  the 
current  operation  of  the  Program,  policies  have  been  identi- 
fied which  are  intended:  to  enhance  the  productivity  of 
Exploratory  Development;  and  to  improve  the  ability  to  demon- 
strate the  need  for,  and  benefits  resulting  from,  the  pro- 
gram's activities  (Ref.  6). 

a.  The  Exploratory  Development  Program  shall  be 
based  on  a set  of  dynamic  technical  strategies 
designed  to: 

(1)  take  maximum  advantage  of  new  technical 
opportunities 

(2)  exploit  deficiencies  in  opposition  capa- 
bilities, and 
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(3)  provide  prompt  response  to  perceived 

requirements  for  superior  naval  capability. 

b.  The  Exploratory  Development  Program  shall  be 
made  visible  through  an  information  system  per- 
mitting technical,  as  well  as  financial  audit 
on  a current  basis  to  cognizant  levels  of 
management.  The  principal  intent  of  the  inform- 
ation system  is  to  permit  control  of  the  tech- 
nical strategy  at  higher  levels,  facilitate 
execution  of  the  programs  at  the  most  techni- 
cally competent  levels,  and  facilitate  the 
transfer  of  technology  to  advanced  development. 

c.  The  Exploratory  Development  Program  shall  include, 
to  the  maximum  extent  practicable , 'block 
programs'  which  are  identified  with  major  tech- 
nical strategies  and  under  the  management  of  an 
organization  whose  mission/product  line  or 
technical  competence  is  most  appropriate. 

d.  The  Exploratory  Development  Program  plan  shall 
be  consistent  with  overall  assigned  ratios  of 
in-house  to  contract  work  for  balanced  utiliza- 
tion of  the  national  technological  base. 

e.  The  Exploratory  Development  Program  and  its 
elements  shall  be  based  on  a more  explicit 
application  of  the  return-on-investment  (ROI) 
concept. 

f.  The  Exploratory  Development  Program  shall  be 
based  on  cost-effective  principles  and  shall 
consider,  as  well  as  budgetary  implications, 

the  potential  costs  of  acquisition  and  ownership. 

g.  The  Exploratory  Development  Program  shall  be 
based  on  providing  adequate  funding  for  effective 
exploitation  of  highest-leverage  opportunities. 
When  over-all  resources  are  inadequate  or 
reduced,  funds  required  for  support  of  such 
projects  shall  be  made  available  through  can- 
cellation or  curtailment  of  projects  with  lower 
promise  of  military  worth  within  the  framework 

of  technical  strategy. 

4 . An  Assessment 

In  general,  the  rationale  and  the  policies  identified 
are  not  inconsistent  with  the  overall  policies  of  Fiscal  and 
Life  Cycles  of  Defense  Systems.  That  is  to  say,  the  basic 
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questions  of  need,  technical  feasibility  and  funding  avail- 
ability are  inherent  in  the  policy  information  for  the 
Exploratory  Development  Program.  However,  an  immediate  con- 
cern which  comes  to  mind  is  whether  the  policies,  being 
quite  broad  and  abstract  by  their  very  nature,  can  be  sup- 
ported and/or  even  interpreted  at  the  working  levels.  This 
concern  is  not  peculiar  to  this  particular  policy  formula- 
tion, rather  it  deals  with  the  structure  of  policy  enactment 
through  any  large  organization. 

Because  the  policies  have  already  been  identified, 
the  approach  that  will  be  taken  now  will  be  to  address  each 
policy  statement  separately  and  provide  personal  comment  as 
deemed  appropriate. 

Policy  A:  Primarily  the  emphasis  should  be  placed 
on  the  term  technical  strategy,  as  a means  of  assuring  that 
the  programs  supported  are  productive,  needed  and  will  pro- 
vide benefit.  It  would  appear  that  a sophisticated  and 
well-managed  communication  (information  flow)  scheme  is 
necessary  to  carry  out  the  desires  of  top  management.  Worth 
questioning  is  the  productivity,  however,  of  focusing  atten- 
tion on  the  deficiencies  of  the  opposition  to  the  detriment 
of  encouraging  an  offensive  approach.  Further,  to  provide 
for  'prompt  response'  would  necessarily  imply  that  the  cor- 
porate memory  and  developmental  capability  of  the  in-house 
sector  be  maintained. 

Policy  B:  Almost  without  exception  would  there  be 
an  agreement  with  the  notion  of  a needed  buildup  of  a more 
meaningful  information  system.  It  is  always  potentially 


25 


possible  though  that  a problem  of  unidirectional  information 
flow  will  ultimately  come  into  being.  There  also  appears  to 
be  some  conflict  with  the  idea  of  maintaining  control  at  the 
higher  levels,  and  at  the  same  time  encouraging  decentraliza- 
tion of  the  EDP . Decentralization  is  essential  in  order  to 
promote  and  produce  technical  excellence  and  thus  realize 
maximum  productivity. 

Policy  C:  There  currently  exists  much  support  of  the 
notion  of  'block  programming'  on  the  basis  of  less  duplica- 
tion of  efforts,  greater  knowledge  of  state-of-the-art  in  the 
particular  product  lines,  and  increased  ability  to  select  and 
evaluate  the  individual  programs  within  the  EDP. 

Policy  D:  Clearly  the  bottom  line  here  is  to  bolster 
and  maintain  a strong  technology  base,  whether  that  be  pro- 
vided for  by  industry  or  in-house.  An  obvious  question  which 
surfaces  given  a predetermined  ratio,  is  just  where  in  the 
organization  that  ratio  must  be  measured — for  this  will  have 
a definite  influence  on  accountability  within  the  system.  It 
should  be  realized  that  the  ultimate  decision  should  be  one 
which  yields  the  maximum  advantage  to  the  government's  carry- 
ing out  its  mission.  This  philosophy  necessarily  requires  a 
consensus  that  those  goals  and  missions  have  properly  and 
adequately  been  delineated. 

Policy  E:  While  the  notion  of  return-on-investment 
appears  relatively  straightforward  at  first  glance,  its  appli- 
cation in  an  area  such  as  Exploratory  Development  is  extremely 
difficult  to  enact.  This  is  due  to  the  obvious  lack  of 
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tangible,  and  thus  quantifiable,  output  from  the  individual 
programs.  However,  in  the  context  of  wanting  to  maximize 
the  benefits  of  the  dollars  invested,  the  concept  is  crucially 
appropriate. 

PoJ icy  P:  As  much  as  is  possible,  attention  should 
be  given  to  the  potential  downstream  costs  of  acquisition 
and  ownership.  It  is  this  basis  which  will  ultimately  provide 
for  a rational  decision  as  to  which  programs  will  be  supported 
in  future  Exploratory  Development  programs. 

Policy  G:  Certainly  the  philosophy  most  often 
followed  in  practice  would  lead  one  to  select  those  alterna- 
tives yielding  the  greatest  leverage.  Seriously  questionable, 
however,  is  that  portion  of  this  policy  that  suggests  each 
program  be  evaluated  on  its  own  merits  and  would  be  subject 
to  cancellation  if  it  shewed  less  favor  in  terms  of  military 
work.  It  is  of  general  belief  (supporting  that  belief  exis- 
ting at  the  NAVMAT  level)  that  in  most  cases,  existing  pro- 
grams will  continue  to  be  funded  unless  there  is  obvious 
reason  to  alter  that  decision  on  the  basis  of  either  lessened 
or  increased  need. 

5.  Concluding  Comments 

There  is  a general  trend  in  the  Navy's  Research  and 
Development  Program,  including  the  Exploratory  Development 
Program,  to  shift  back  to  an  originally  supported  organiza- 
tional structure  of  being  decentralized.  This  is  due  to  a 
large  extent  to  the  desire  to  promote  accountability  at  all 
levels  of  the  R&D  organization.  Additionally,  it  is  observed 
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that  trends  definitely  indicate  future  total  budgets  to  rise, 
while  the  percentage  allocation  to  the  Exploratory  Develop- 
ment category  will,  as  has  been  experienced  in  the  past 
decade,  definitely  decline.  As  a result  of  this  observation, 
it  is  noted  that  technology  development  is  definitely  not 
independent  of  weapon  system  development. 

It  is  noteworthy  in  summary  to  exploit  the  under- 
lying principles  evidenced  in  the  policy  formulation  for  the 
Exploratory  Development  Program: 

• Supportive  of  technological  advances  of  funda- 
mental importance  to  operational  needs. 

• Quest  for  technical  and  managerial  excellence  at 
all  levels. 

• Recognition  of  the  necessity  for  both  top-down 
direction  and  bottom-up  inputs. 

• Accountability  essential  at  all  executive  levels. 

• Necessity  for  strategies  as  tools  to  implement 
policies. 

• Recognition  that  sometimes  technical  advance  must 
be  injected  into  the  mainstream  of  the  system 
forcibly. 

• Recognition  of  essential  need  for  adequate 
information  systems  to  allow  interaction. 

While  it  is  a rather  straightforward  task  to  outline 
a rational  approach,  it  is  extremely  difficult,  if  not 
impossible,  to  encounter  a purely  rational  organization  in 
operation.  Influences,  personalities,  and  constraints  are 
entirely  too  numerous  to  allow  anything  other  than  some  form 
of  an  "individualized"  organization  to  exist  today,  whether 
that  organization  be  public,  private  or  some  mix  of  the  two 
extremes . 
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An  investigation  of  the  EDP  policy  does,  however, 
serves  to  provide  the  foundation  on  which  its  organization 
must  be  structured.  Further,  this  foundation  (rationale 
and  policy)  provides  the  impetus  behind  the  current  EDP  mod- 
ification, which  is  the  subject  matter  of  the  following 
section  of  this  thesis.  It  is  important  that  the  reader  be 
aware  that  in  addition  to  gaining  a better  understanding  of 
the  "real"  system  in  operation  today,  the  material  presented 
will  hopefully  provide  a context  in  which  to  review  the  sub- 
jects of  program  selection  and  evaluation  in  applied  tech- 
nology in  a more  meaningful  manner. 

B.  STRUCTURE  OF  EDP  (A  CHRONICLE) 

The  creation  of  the  offices  of  the  Director,  Defense 
Research  and  Engineering  (DDR&E) , Assistant  Secretary  of  the 
Navy  (Research  and  Development) , and  Deputy  Chief  of  Naval 
Operations  (Development)  (DCNO(D))  back  in  1959  set  the  stage 
for  some  revolutionary  changes  in  the  way  R&D  programs , and 
hence  Exploratory  Development  programs,  would  be  planned, 
justified  and  implemented  for  the  next  two  decades.  The 
concept  of  a complete  R&D  spectrum  with  basic  research  on  the 
one  end  and  full  scale  development  on  the  other  was  inherent 
in  the  procedures  of  the  first  decade. 

Technology  programs  were  established  which  covered  a wide 
variety  of  efforts,  ranging  from  pure  research  in  components 
and  subsystems  to  fairly  sophisticated  experimental  hardware. 
These  technology  programs  at  that  time  were  characterized  as 
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being  comprised  of  projects  more  closely  focused  on  end- 
products  than  research  but  entailing  a higher  level  of  uncer- 
tainty and  lower  financial  commitment  than  full  scale 
development. 

Bureaus  and  offices  were  established  in  areas  relevant  to 
their  assigned  responsibilities.  Project  listings  were 
required  to  be  structured  according  to  the  format  of  Table  I 
(Ref.  7) . So  for  the  first  time,  senior  staffs  in  the  Depart- 
ment of  the  Navy  and  the  Office  of  the  Secretary  of  Defense 
had  a viable  framework  for  scrutinizing  the  exploratory  devel- 
opment effort  and  the  funds  allocated  to  it. 


Table  I Exploratory  Development 
Functional  Areas,  1962 


F001 

Target  Surveillance 

F002 

Navigation 

F003 

Environmental  Surveillance 

F004 

Integrated  Surveillance 

F00  5 

Command  Control 

F006 

Communications 

F007 

Data  Processing 

F008 

Weapons  and  Ordnance 

F009 

Guided  Missiles 

F010 

Jamming  and  Deception 

F011 

Naval  Defense  Applications 

F012 

Aircraft  and  Aircraft  Support 

F013 

Ships  and  Submarines 

F014 

Boats  and  Amphibious  Vehicles 

F015 

Logistics 

F016 

Personnel  Administration 

F017 

Training 

The  process  of  planning  and  justifying  programs  in  the 
early  1960s  is  broadly  depicted  in  Figure  1 (Ref.  7) . There 
was  a trend  in  the  early  years  for  the  bureaus  and  offices  to 
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employ  a "carte  blanche"  attitude  to  initiate  new  programs 
in  response  to  general  requirements.  This  philosophy  began 
to  erode,  however,  with  the  enactment  of  PPBS , and  eventually 
the  organization  became  extremely  controlled,  and  hence,  cen- 
tralized by  the  end  of  the  first  decade. 

In  1970,  DOR&E  then  introduced  two  new  types  of  documents 
which  more  clearly  stressed  the  concern  of  program  worth, 
justification  and  relationship  to  mission  areas.  They  were 
called  Technology  Coordinating  Papers  and  Area  Coordinating 
Papers.  They  were  intended  to  highlight  program  gaps,  over- 
laps, and  need  for  improved  coordination  among  the  Services 
(Ref.  8)  . 

And  so,  during  the  early  baseline  period,  technology  pro- 
grams represented  a segment  of  the  Navy ' s R&D  program  in 
which  the  management  operated  with  little  or  no  outside  inter- 
ference. Guided  only  by  the  most  general  requirements,  bureau 
personnel  relied  heavily  on  Fleet  and  laboratory  personnel, 
discussions  with  the  Fleet  and  experience  of  their  own  offi- 
cers and  engineers  to  identify  problems,  initiate  programs 
and  evaluate  their  on-going  projects. 

There  was  a trend  toward  greater  emphasis  and  dependence 
on  the  Navy  laboratories,  i.e.,  block  programming.  The  bureau/ 
SYSCOM  point  of  view  initially  was  to  oppose  block  programming 
on  the  grounds  that  it  amounted  to  delegating  authority  to 
the  laboratories  without  commensurate  accountability  for  ful- 
filling SYSCOM  responsibilities.  The  idea  of  block  programming 
has  since  become  a reality,  and  is  now  in  full  effect,  with 
nearly  full  support,  in  the  Navy  organization. 
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Block  programming , as  interpreted  today,  is  a means  for 
providing  technical  management  in  the  laboratories  and  Systems 
Commands  with  the  necessary  authority  (including  control  of 
funds)  to  plan  and  manage  technical  effort  related  to  the 
pursuit  of  significant  objectives.  As  will  be  seen  later, 
block  programming  is  a significant  influence  in  the  current 
planning  and  justification  process  proposed  for  the  Explora- 
tory Development  Program. 

C.  A CHANGING  CONTEMPORARY  PROGRAM 

The  Navy  Exploratory  Development  Program  is  just  nearly 
completing  a transition  from  a system,  which  has  been  in 
operation  for  a number  of  years  and  is  relatively  well  known 
by  the  Navy  R&D  community,  to  a new  system.  Central  to  these 
modifications  is  the  use  of  dynamic  "technical  strategies," 
designed  to  focus  exploratory  development  more  directly  and 
visibly  on  Department  of  the  Navy  operational  needs.  The 
first,  implementation  of  the  program  utilizing  these  technical 
strategies  will  commence  during  the  current  fiscal  year. 

This  program  reorganization  is  a result  of  the  CND/DCNM(D) 
having  been  directed  by  the  ASN  (R&D)  to  establish  and  imple- 
ment a program  to  improve  the  exploratory  development  system, 
to  improve  higher  echelons'  understanding  of  the  system  and 
to  justify  6.2  expenditures. 

As  indicated  previously,  under  the  former  6.2  management 
system  the  program  consisted  of  a large  number  of  low  level 
funded  tasks  which  were  negotiated  primarily  between  staff 
personnel  of  the  SYSCOMs  and  the  laboratories  without 
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direction  of  either  top  level  management  or  the  best  qualified 
technical  personnel  who  are  generally  located  at  the  labora- 
tories. The  relationship  between  funded  tasks  and  operational 
needs  was  not  altogether  visible  and  the  contributions  that 
new  technologies  could  make  to  increase  effectiveness  or 
decrease  costs  were  not  apparent.  As  a result,  justification 
of  the  program  to  Congress,  OSD  and  to  ASN  (R&D)  was  not  con- 
vincing. There  was  a lack  of  funding  focus  on  high  payoff 
innovation  and  on  feasibility  demonstration  of  critical  end 
item  products.  Further,  there  had  been  an  inadequate  inter- 
action of  6.1  with  6.2  and  coupling  with  6.3  and  a lack  of 
coordination  with  related  work  by  other  services  and  industry. 

The  objectives  of  the  modified  6.2  management  system  were 
then  identified  to  (Ref.  9) : 

• Provide  top-down  direction  by  the  use  of  technical 
strategies,  prioritized  technical  objectives  and 
definition  of  major  program  thrusts  designed  to  meet 
operational  requirements. 

• Provide  visibility  by  relating  the  program  to  Navy 
functions  and  to  operational  needs . 

• Promote  innovation  and  concentration  on  highest 
leverage  opportunities. 

• Expedite  transfer  of  projects  to  advanced  development 
(6.3)  and  use  of  new  technical  capabilities  developed 
in  research  (6.1). 

• Demonstrate  relevance  and  cost  effectiveness. 

These  objectives  were  to  be  achieved  by  clear  definition 
of  organizational  responsibilities,  development  and  execution 
of  programs  to  achieve  approved  technical  objectives,  allo- 
cation of  blocks  of  funds  to  organizations  and  delegation  of 
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authority  to  implement  programs  to  meet  assigned  responsibili- 
ties. CND  is  the  designated  responsible  entity  to  ASN  (R&D) 
for  operation  of  the  system.  Appendix  A is  included  to  delin- 
eate the  EDP  management  responsibilities,  including  design 
principles  and  roles  of  the  various  organizations  in  the  6.2 
Program. 

There  are  four  basic  elements  of  the  new  6.2  planning  sys- 
tem which  utilize  the  technical  strategy  concept. 

1 . Direction  by  Technical  Strategy 

Designated  strategists  identify,  define  and  analyze 
mission  needs  and  on-going  or  potential  technical  efforts 
related  to  satisfying  those  needs.  Mission  needs  are  derived 
from  analyses  of  threat,  preferred  operational  concepts  and 
system  and  subsystem  concepts.  Current  programs  and  work 
by  industry  and  other  services  are  major  factors  in  assessing 
technical  potential.  Technical  objectives  are  derived  from 
mission  needs  and  potential  technology.  The  strategies  then 
are  used  to  define  the  desired  major  thrust  in  prioritizing 
the  technical  objectives. 

These  dynamic  technical  strategies  are  designed  to 
take  maximum  advantage  of  new  technological  capabilities, 
exploit  deficiencies  in  opposition  capabilities  and  provide 
prompt  response  to  perceived  requirements  for  superior  naval 
capabilities  (Ref  6).  Basically,  strategic  planning  is  not 
a new  idea,  and  it  implies  "...  the  process  of  determining 
the  major  objectives  of  an  organization,  and  the  policies  and 
strategies  that  will  govern  the  acquisition,  use  and  disposi- 
tion of  resources  to  achieve  those  objectives"  (Ref.  10) . 
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The  enactment  of  an  Exploratory  Development  program 
policy  based  on  the  idea  of  technology  strategies  is  a signif- 
icant breakthrough  in  terms  of  developing  a more  meaningful 
rationale  for  program/project  evaluation.  The  primary  reason 
for  this  is  that  heretofore,  organizations  had  neglected  to 
recognize  that  there  had  been  a definite  lack  of  clarity  in 
just  what  they  were  trying  to  measure.  This  belief  is  sup- 
ported by  various  studies  of  productivity  measurement  and  pro- 
gram evaluations  (Ref.  11) . 

It  should  be  recognized  that  Exploratory  Development, 
like  every  other  activity,  must  compete  for  its  resources. 
Although  the  necessity  for  an  extensive  program  is  obvious  to 
those  within  the  RDT&E  establishment,  to  assure  maintenance 
at  the  desired  level,  the  program  must  continually  be  justi- 
fied in  meaningful  and  convincing  terms.  To  assure  that  long 
term  benefits  are  clear,  Exploratory  Development  thus  must 
demonstrate  how  it: 

a.  provides  opportunities  which  stimulate  the 
development  of  improved  or  totally  new 
operational  capabilities , 

b.  provides  solutions  to  problems  arising  in 
the  development  process, 

c.  provides  opportunities  to  regain  operational 
superiority  or  enhance  existing  superiority, 

d.  provides  for  the  transformation  of  new  research 
discoveries  into  useful  applications,  and  the 
extension  of  existing  technologies  into  signifi- 
cantly more  advanced  applications. 

To  accomplish  this,  then,  the  system  being  assembled 
at  the  NAVMAT  03  level  addresses  not  only  objectives,  strate- 
gies and  tactics  (OST) , but  also  interrelates  them  and 


36 


describes  how  the  system  works  to  harness  research  and  devel- 
opment for  the  achievement  of  definite  organizational  goals 
(Ref.  12) . 

Essentially  the  OST  system  is  based  on  three  major 
ideas:  (1)  formulation  of  an  OST  plan  consisting  of  a hier- 

archal  arrangement  of  corporate  objectives,  technical  strate- 
gies, and  developmental  tactics,  (2)  the  super  position  of 
this  organization  over  the  normal  operating  organization, 
and  (3)  the  use  of  line  managers  in  the  operating  organiza- 
tion as  action  officers.  Figure  2 presents  this  system  in  a 
somewhat  simplified  manner. 

This  current  model  thus  contains  all  the  elements  of 
the  historical  and  modern  approaches,  e.g.,  cost,  timeliness, 
risk,  weapon  systems,  value  measures,  etc.,  and  also  provides 
the  additional  and  quite  significant  notion  that  the  goals 
and  policies  of  the  organization  need  be  accounted  for. 

The  strategies  then,  as  developed  for  the  Navy  EDP, 
will  indicate  allocations  recommended  for  additional  objectives 
with  justification.  DNT  will  provide  guidance  regarding  over- 
all concepts  for  functional  areas,  and  also  to  strategists 
through  the  inclusion  of  relevant  material  from  program  objec- 
tive memorandums . 

2.  Negotiating  the  Program 

With  the  implementation  of  the  new  policy,  all  explor- 
atory development  programs  will  now  be  prenegotiated  between 
SYSCOMs , laboratories  and  DNT  prior  to  beginning  each  fiscal 
year.  After  the  programs  have  been  developed  they  then  are 
recommended  to  DNT  for  allocation  of  funds. 
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OST  STRUCTURE 


There  are  three  types  (categories)  of  programs,  the 
characteristics  of  which  determine  how  the  organization  will 
manage  them.  Cat  I is  broadbased,  multi-use  technology.  It 
is  negotiated  between  DNT  and  laboratory  CO/TDs  and  funded 
and  managed  by  the  laboratories.  All  "special  topic"  strategy 
areas  are  included  in  Cat  I programs.  Cat  II  is  concept- 
related  effort  for  which  the  SYSCOMs  are  customers.  It  is 
concerned  primarily  with  exploratory  development  of  systems 
and  subsystems  concepts  for  accomplishing  mission  needs. 
Projects  for  this  kind  of  work  are  negotiated  between  the 
using  SYSCOMs  and  the  cognizant  laboratories.  Once  negotiated 
and  approved  by  DNT,  it  is  funded  directly  as  a block  to  the 
performing  laboratory  which  manages  it.  Cat  III  work  is  per- 
formed primarily  by  industry  or  other  contractors.  It  in- 
volves primarily  work  related  to  transition  of  concepts  to 
advanced  development,  and  it  is  managed  directly  by  the  SYS- 
COMs. Figure  3 shows  the  "chain  of  accountability"  which  now 
exists  in  the  EDP  and  the  types  ( categories ) or  programs  for 
which  the  various  tiers  are  responsible.  The  planning  pro- 
cess which  has  been  formulated  in  consert  with  the  new  manage- 
ment system  is  depicted  in  Figure  4. 

3 . Allocation  of  Funds 

Of  the  total  funds  allocated  for  the  EDP,  the  Chief 
of  Naval  Development  withholds  a total  of  15  percent  of  each 
program  element  to  take  care  of  subsequent  Congressional  cuts, 
to  provide  for  new  starts,  for  developing  the  strategies,  for 
IR/IED,  MAT  03  management,  the  Energy  Office  and  DNT's  special 
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I - BROAD  BAND  TECHNOLOGY 
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projects.  The  bulk  of  the  funds  (roughly  $2.75B  yearly) 
are  allocated  to  SYSCOMs  and  laboratories.  Once  the  con- 
tracts for  work  have  been  negotiated,  changes  can  be  made 
only  by  the  SYSCOMs  for  the  CAT  III  programs  and  TD  of  labor- 
atories for  CAT  I and  II  programs.  Final  approval,  as  before, 
always  rests  with  the  Director  of  Navy  Technology. 

4 . Review  and  Evaluation 

The  review  and  evaluation  focuses  on  two  major 
issues:  (a)  were  the  resources  actually  utilized  as  covered 

in  the  approved  program  as  amended  and  (b)  to  what  degree 
were  the  technical  objectives  developed  by  the  strategies 
accomplished.  It  was  perceived  by  management  that  the  basic 
review  and  evaluation  questions  should  focus  on  contributions 
to  future  Navy  operations  and  cost  ef fectivensss . At  the 
time  of  research  gathering  for  this  paper  there  were  no  de- 
tails available  as  to  how  the  review  would  be  conducted  but 
the  objective  is  clear — substantive  evaluation  from  the  view- 
point of  Congress,  DDR&E  and  ASN  (R&D)  and  not  micromanagement 
It  is  anticipated  by  EDP  management  that  program  administra- 
tors, strategists,  and  a qualified  panel  of  independent 
individuals  with  knowledge  of  Navy  needs,  and  other  Federal 
and  industrial  efforts  would  be  involved  in  review  and  evalu- 
ation process.  However,  it  should  be  noted  again  that  no 
clear  and  definitized  methodology  yet  exists  to  conduct  a 
complete  program  selection,  review  and  evaluation  program 
(Ref.  9) . 
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5.  Operation — Correlation  of  Elements  and  Strategies 

In  development  of  the  strategies,  the  program  element 
titles,  which  had  been  originally  listed  by  the  CND,  were 
expanded  to  the  current  total  of  37  (Ref.  9) . Although  this 
increase  facilitated  submission  by  the  cognizant  internal 
agency,  its  effect  is  to  fracture  the  program  and  make  more 
difficult  their  aggregation  in  an  overall  plan.  Figure  5 
shows  the  relationship  which  now  exists  between  program  ele- 
ments (PEs) , technical  strategy  areas  and  the  Science  and 
Technology  Objectives  (STOs) . It  is  noted  that  there  is  a 
high  degree  of  correlation  between  PEs  and  strategies  except 
in  sea  control  and  power  projection  where  the  strike  warfare, 
countermeasures  and  two  other  program  elements  are  split  be- 
tween sea  control  and  projection. 

The  Chief  of  Naval  Development  establishes  and  promul- 
gates the  structural  framework  for  planning,  programming  and 
budgeting  the  EDP  (Appendix  B) . This  structure  provides  the 
link  with  DoD  Program  elements,  defines  the  scope  of  work 
within  specific  areas,  and  identifies  the  organization  within 
the  Navy  responsible  for  the  planning,  programming  and 
budgeting.  Appendix  C is  included  to  provide  clarification  of 
the  fundation  of  the  Exploratory  Development  program  by  delin- 
eating its  new  structural  framework. 

The  system  described  above  is  new  this  current  fiscal 
year.  It  is  quite  naturally  going  to  involve  a great  deal  of 
"negotiating"  between  the  staffs  of  NAVMAT , laboratories  and 
SYSCOMs,  along  with  DNT,  CND,  and  ASN  (R&D) . It  is  essential 
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that  a firm  understanding  of  the  technical  strategies,  func- 
tional areas  and  program  elements  be  had  to  augment  an  under- 
standing of  the  EDP  objectives  and  transition  steps.  Only 
through  a thorough  knowledge  of  the  inner-workings  of  the  new 
system  will  any  meaningful  review,  evaluation  and  assessment 
methodology  be  realized.  As  indicated,  no  established  pro- 
gram or  technique  currently  exists,  but  there  certainly  is 
potential  for  developing  a viable  methodology,  particularly 
with  the  well  structured  and  goal-oriented  EDP  which  exists 
today . 


IV.  PROGRAM  SELECTION  AND  EVALUATION  IN  NAVY 


EXPLORATORY  DEVELOPMENT 

With  a background  presented  of  the  Navy  Exploratory 
Development  program,  its  functions,  its  responsibilities  and 
its  policies  for  implementation,  it  is  beneficial  to  investi- 
gate now  the  manner  in  which  programs  have  traditionally  been 
reviewed,  selected  and  evaluated.  It  would  serve  the  reader 
well  to  keep  in  mind  the  new  EDP  characteristics,  particularly 
while  going  through  this  review.  This  will  provide  a deeper 
comprehension  of  why  the  methodologies  to  be  discussed  were 
considered  viable,  as  well  as  why  they  eventually  were  evi- 
denced as  not  being  able  to  adequately  predict,  as  a manage- 
ment tool,  as  originally  intended. 

A.  BACKGROUND 

Technology  base  programs  constitute  a large  number  of 
projects  in  the  systems  acquisition  process,  falling  some- 
where between  pure  research  on  one  end  of  the  R&D  spectrum 
and  full  scale  development  on  the  other.  In  the  early  years 
of  the  Navy  R&D  sector,  these  technology  programs  operated 
with  little  or  no  outside  interference  to  the  performing 
organization.  Ideas  for  solving  Fleet  related  problems  and 
exploiting  technological  opportunities  were  converted  into 
exploratory  development  projects  with  relative  ease  (Ref.  7) . 

In  these  early  stages  of  R&D  project  plans,  task  assign- 
ments and  monitoring  procedures  were  generally  flexible  and 
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tailored  to  the  needs  of  the  individual  managers  at  all 

{ 

levels.  This  process  was  in  keeping  with  the  "cradle  to 
grave"  concept. 

More  recently,  a perceived  need  to  provide  guidance  for 
technology  programs  was  reflected  in  documents  ranging  from 
top  level  Joint  Chiefs  of  Staff  strategic  planning  documents 
to  OPNAV's  General  Operational  Requirements  and  CND  Explora- 
tory Development  Goals.  But  this  guidance  soon  tended  to 
be  either  too  general,  providing  little  guidance  at  all,  or 
so  specific  that  it  inhibited  exploration  of  relevant 
alternatives . 

Because  the  problems  associated  with  this  guidance  and 
direction  quite  naturally  are  felt  at  all  levels  within  the 
organizational  structure,  the  problem  of  effective  program 
selection  and  evaluation  has  thus  remained  a significant 
issue,  particularly  in  the  area  of  exploratory  development. 

According  to  a recent  House  Armed  Services  Committee 
report,  concern  was  expressed  that  "...  just  spending  money 
on  research  and  development  does  not  guarantee  a sound  R&D 
program  or  good  military  hardware"  (Ref.  13) . The  committee 
further  indicated  that  the  Department  of  Defense  could 
improve  its  efforts  and  its  management  through  better  tailored 
programs  which  would: 

• Select  out  weapons  not  cost  effective 

• Avoid  unnecessary  duplication  of  effort 

• Place  greater  emphasis  on  product  improvement 

• Provide  more  meaningful  management  review  processes. 
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It  is  not  surprising  that  a tremendous  amount  of  attention 
is  currently  being  given  to  the  valuation  of  DoD  programs 
in  Research  and  Development  in  general  and  Exploratory  Devel- 
opment in  particular. 

Adding  now  to  the  complexity  of  this  situation  is  the  fact 
that  throughout  the  past  fifteen  years,  while  the  total  budget 
for  RDT&E , N appropriations  has  been  steadily  increasing,  the 
relative  percentages  allocated  to  the  technology  base  pro- 
grams (6.2  category)  have  been  steadily  decreasing.  This 
situation  is  depicted  in  the  bar  chart  representation  of 
apportionments,  shown  as  Table  II. 

And  so,  just  within  the  past  decade,  many  scientific,  and 
some  not  so  scientific,  approaches  have  been  tried  for  poten- 
tial application  to  technology  base  program  evaluation  and 
determination  of  return-on-investment  figures.  Most  of  these 
scientific  techniques  have  been  tried  more  at  the  lower 
echelons  of  the  Navy  R&D  organization,  although  few  have  seen 
any  real  application  or  effectiveness.  The  approach  currently 
in  vogue  at  the  NAVMAT  03  level  is  built  around  the  concept 
of  "technology  strategies.”  There  appears  to  be  merit  in 
pursuing  this  technique,  at  least  for  a selected  application. 

B.  HISTORICAL  APPROACHES 

One  of  the  early  comprehensive  attempts  to  sort,  categorize 
and  evaluate  R&D  selection  and  program  evaluation  techniques 
was  that  done  by  Baker  and  Pound  back  in  the  mid-sixties  (Ref. 
14) . This  article  points  out  that  most  were  oriented  toward 
the  operations  research-management  scientist,  but  it  does 
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Table  II 


Research  and  Development  Categories  as 
Percent  of  Total  RDT&EJM  Appropriation  FY62-FY73 


IM2  1961  1M4  IMS  19 


M 1M7  IMS  IMS  1970  1971  1977  1973 

FISCAL,  YtAR  PROGRAM 


Not*:  Those  amounts  art  derived  from  the  past  y*ar  column*  of  the  various  Congressional 
Budget  Subm.ssrons  and  »'•  not  prae Italy  contistant  with  Now  Obligation*!  Authorizations 
Tha  diffaranc*  results  from  reprogramming  between  program  years,  permitted  in  "no-yeer” 
appropriations  The  data  for  tech  fiscal  year's  program  it  the  latest  category  distribution 
developed  far  budget  presentation  and  best  reflects  actual  relative  dollar  size  of  the  various 
categories  for  each  fiscal  year.  FY  1962  mm  the  fir^  yea ' in  which  this  category  classifica- 
tion schema  was  systematically  applied. 

Source.  Thomas  Lackey,  Assistant  Comptroller  for  Budget  and  Reports  OMR.  ,n  letter  o f 
34  Oeeember  1974. 


49 


provide  a sound  base  for  presenting  the  early  methods  thought 
to  have  potential.  It  is  interesting  to  note  that  a review 
of  the  majority  of  the  documents  reviewed  in  this  account 
covering  this  early  era  indicated  that  very  few  of  the  tech- 
niques presented  actually  ever  saw  any  relevant  usage. 

Keeping  in  consonance  with  the  early  organization  and 
management  of  the  exploratory  development  sector  of  R&D,  pro- 
ject selection  thus  remained  a relatively  unstructured  prob- 
lem. For  the  most  part,  the  early  methods  just  did  not 
provide  an  adequate  description  of  the  R&D  process.  At  this 
point,  it  might  appear  feasible  to  question  not  only  the  pre- 
dictive models  but  the  process  which  the  models  were  attemp- 
ting to  simulate.  While  it  was  recognized  that  program 
selection  was  a sequential  decision  making  process,  practi- 
cality would  certainly  indicate  that  detailed  criticism  of 
these  early  models  would  be  unwarranted. 

One  important  observation  made,  according  to  Baker,  was 
that  "...there  has  not  been  a satisfactory  definition  of  the 
objectives  against  which  a research  program  should  be  evalu- 
ated." The  significance  of  this  statement,  while  having 
clarity  at  this  point,  will  take  on  new  interpretations  later 
in  this  section.  Another  key  issue  raised  in  these  early 
studies,  although  it  received  little  more  attention  than  having 
been  mentioned,  was  the  idea  that  the  "strategic  need"  some- 
how ought  to  fit  into  the  organizational  and  market  structure. 

The  methods  applied  in  the  early  stages  of  investigation 
consisted  almost  entirely  of  three  major  methods:  decision 


50 


theory,  economic  analysis  and  operations  research.  Table  III 
is  included  in  an  attempt  to  summarize  the  significant  features 
of  these  methodologies. 


Table  III 


Features  of  Common  R&D 

Planning  Models 

Feature 

Method 

Characteristic  consideration 

Decision 

theory 

Economic 

analysis 

Operations 

research 

1. 

Sequential  decisions 

No 

No 

Yes 

2. 

Profit  maximization 

Partially 

Yes 

Yes 

3. 

Uncertainty  (probability 
of  success) 

Yes 

Yes 

Yes 

4. 

Relationships  between  criteria 

No 

Partially 

Yes 

5. 

Relationships  between  projects 

NO 

No 

Partially 

6. 

Optimal  allocation  to  project 

NO 

Partially 

Yes 

7. 

Budget  constraint 

Yes 

Yes 

Yes 

8. 

Skill  & facility  constraints 

No 

NO 

No 

9. 

"Strategic  need"  for  project 

Partially 

NO 

No 

10. 

Competitor  efforts 

No 

NO 

No 

Area  of  applicability 


1. 

Basic  research 

Yes 

Maybe 

No 

*2. 

Applied  research 

Maybe 

Yes 

Maybe 

3. 

Development 

Little 

Moderate 

Definitely 

♦Technology  base  programs. 

A prime  observation  of  these  methods  and  practices  is  that 
there  exists  a strong  and  definite  need  for  testing  to  demon- 
strate feasibility  of  the  predictive  models.  Without  adequate 
demonstration,  even  the  "best"  descriptive  models  prove  sus- 
pect under  the  close  scrutiny  of  management. 
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C.  MODERN  APPROACHES 

Once  the  idea  that  the  management  process  of  project 
selection  and  evaluation  was  realized  to  have  potential  as 
being  predictable  from  a management  scientist  approach,  a 
profuse  amount  of  literature  emerged,  quite  naturally.  A 
great  many  new  ideas  were  postulated,  some  unique  in  their 
own  right,  although  most  were  merely  extensions  of  what  had 
already  been  presented  in  the  early  years. 

An  excellent  study  conducted  by  IDA  provides  an  informa- 
tive synopsis  of  the  modern  techniques  being  employed  (Ref. 
15).  Although  prepared  in  early  1972,  the  methods  discussed 
are  still  being  utilized,  if  only  under  slight  improvement. 

Because  the  modern  techniques  have  tended  to  align  them- 
selves in  the  more  strict,  quantitative  approach  to  program 
selection  and  evaluation,  most  of  the  discussion  at  this 
point  will  tend  to  follow  along  with  that  momentum.  With  the 
ensuing  budget  constraints,  as  previously  mentioned,  the 
patter  associated  with  the  modern  approaches  will  necessarily 
allude  to  resource  allocation  as  its  primary  objective. 
Realize,  however,  that  the  real  issue  is  still  the  process  of 
determining  which  programs  are  to  be  selected  for  support  and 
how,  then,  can  they  be  evaluated  once  initiated. 

Some  problems  are  inherently  embedded  in  the  modern 
methodologies,  owing  to  their  strong  alliance  to  the  quanti- 
tative approach  of  evaluation.  A strong  underlying  problem 
is  that  even  with  modern  conveniences  and  sophisticated  ana- 
lytical software  and  hardware,  the  utilization  factor  still 
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remains  low  (Ref  16) . According  to  Albala  in  his  work  on 
stage  approaches  to  R&D  projects: 

This  condition  can  be  partly  ascribed  to  the  fact 
that  (a)  most  of  the  proposed  methods  involve  models 
requiring  much  quantitative  input,  not  readily  avail- 
able within  the  company,  and  (b)  in  many  instances 
the  methods  are  based  on  techniques  much  too  elaborate 
for  the  R&D  manager's  routine  use. 

In  subtle  opposition  to  this  belief  is  the  posture  taken 
by  Trozzo  of  IDA  (Ref.  15.)  Accordingly,  Trozzo  feels  that: 

Development  of  more  quantitative  methods  for  allo- 
cating resources  within  Exploratory  Development  should 
continue,  however.  Decause  their  rigorous  structure 
should  provide  (1)  a useful  frame  of  reference  for 
organizing  information  and  regular  liaison  among  system 
specialists,  technologists,  and  research  managers,  and 
(2)  a systematic  procedure  for  searching  out  a develop- 
ment program  that  will  best  fulfill  Defense  goals. 

Albeit,  included  as  Table  IV  a summary  description  of  the 
methods  used  for  allocating  resources,  taken  directly  from 
the  study  by  IDA.  Its  inclusion  is  useful  in  showing  the 
definite  trend  to  quantitative  rationale  and  also  to  demon- 
strate that  each  organization  utilizes  a methodology  unique  to 
its  particular  operation,  based  to  a large  extent  on  its  own 
needs  and  variables. 

Thus  the  techniques  that  have  grown  to  occupy  currently 
acceptable  standards  have  adhered  to  the  general  conclusions 
of  the  earlier  studies.  To  wit,  there  is  a general  recognition 
that  each  particular  stage  of  R&D  requires  a set  of  inputs  and 
constraints  unique  to  its  decision  making  process  (Ref.  16) . 
Another  important  observation  of  modern  approaches  is  their 
recognition  that  models  need  be  commensurate  with  the  quality 
and  quantity  of  data  available.  Finally,  project  success,  to 
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Table  IV 

Summary  Description  of  Methods  for 
Allocating  Development  Resources. 


Primary  Objective 

_ 

Value  Measures 

Operational 

Requirements 

Method 

Devise  an  exploratory 
development  program 
that: 

Management's  levers 

Usefulness  or  con- 
tribution of  program 
results 

Major  jobs  of  the 
Services 

1. 

Industrial 

Analog 

Resembles  the  pattern 
of  technological  de- 
velopment in  U.S.  in- 
dustry 

Funds  spent  in  major 
technical  areas 

None 

Industry  groupings — 
weapon  categories 

2. 

TORQUE 

Develops  technologies 
having  maximum  "mili- 
tary value" 

Funds  to  be  spent  in 
next  year  on  project 
tasks 

Subj active  scaling  of 
relative  "military 
va lue" 

Operational  capa- 
bility objectives 
(OCO's)  assigned 
values 

3. 

Naval  Ordnance 
Laboratory 

Develops  tectnologies 
having  maximum 
"value"  for  future 

Navy  needs 

Funds  to  be  spent  on 
each  technology  in 
next  year 

Subjective  scaling  of 
relative  value  for 
each  factor 

Warfare  categories  — 
targets — military 
functions  assigned 
values 

4. 

A.F.  Flight 

Dynamics 

Laboratory 

Develops  technologies 
having  maximum  "mili- 
tary value"  for  the 
Air  Force 

Funds  and  manpower  to 
be  devoted  to  each 
exploratory  develop- 
ment task 

Subjective  scaling  of 
relative  effective- 
ness, timing,  etc. 

Operations  within 
different  types  of 
wars — assigned  values 

5. 

Cornell 

Aeronautical 

Laboratory 

Consists  of  tasks 
with  maximum  "ex- 
pected value"  to  the 
Army 

Tasks  that  will  be 
performed 

"Essentiality"  of 
weapons  developed 

No  general  opera- 
tional requirements. 
Background  to  qual- 
itative materiel 
development  objec- 
tive CQMDO) 

6. 

Hercules 

Corporation 

Contributes  most  to 
future  company  profits 

Funds  to  be  spent  on 
individual  develop- 
ment tasks 

Profits  generated  by 
development  work 

Market  characteris- 
tics 

7. 

Army  Missile 

Plan 

Supports  preferred 
weapon  concepts  pro- 
jected for  future 
deployment 

Weapons  and  funding 
of  requisite  tasks 

Discrete  priorities 
among  weapons  and 
technical  work 

Operational  require- 
ments in  documents  by 
CDC 

9. 

A.F.  Directorate 
of  Laboratories 
PLan 

Provides  Che  technol- 
ogy required  for  prob- 
able future  systems 

Technical  effort  in 

each  area 

None 

Desired  capabilities 
defined  by  OSD  and 
the  Air  Staff 

9. 

Army  Research 

Plan 

Responds  to  long 
range  concepts  and 
objectives  of  the 

Army  of  the  future 

Funding  emphasis  for 
elements  of  program 

Priorities 

Future  operational 
needs  in  56  opera- 
tional capability  ob- 
jectives 

10. 

Another  Service 
Method 

Supplies  technical 
advances  for  ful- 
filling overall 

Service  mission 

Funding  of  individual 
development  projects 

Subjective  scaling  of 
merit  of  requirements 
and  contribution  of 
technical  objectives 

29  operational  re- 
quirements—merit 
weights 

«? 
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Table  IV  (Cont'd) 


Structural  Features  and  Factors  Considered 


Factors  and  Their  Relationships 


Weapon  Systems 

technologies 

Development  Tasks 

Costs 

Method 

Instruments  for  per- 
forming jobs 

Composition  of 
systems 

work  to  achieve  de- 
sired composition 

Resources  consumed  in 
work 

1.  Industrial 

Analog 

Individual  weapons — 
industry  products 
(DoD  expenditures — 
industry  sales) 

Weapon-product  life- 
times and  technical 
complexity 

None 

DoD  expenditures  in 
6.1,  6.2  and  total 
RDTtE  on  technical 
field-company  funded 
RSD  in  industry 

2.  TORQUE 

Alternative  weapons 
for  each  OCO 

Technological  re- 
quirements of  each 
weapon — criticality 
to  system  rated 

Technical  objectives 
sequenced  in  each 
technical  require- 
ment 

Yearly  and  total 
funding  for  each 
project  task 

3.  Naval  Ordnance 
Laboratory 

Contained  in  explor- 
atory development 
goals  (EDG)  military 
worth 

Technology  pacing 
parameter  linked  to 

EDG  by  relevance 
measure 

Implicit — different 
levels  of  pacing 
parameters 

Aggregate  expendi- 
tures on  technology 
related  to  index  of 
advance  for  parameter 

4.  A.F.  Flight 
Dynamics 
Laboratory 

Future  A.F.  flight 
vehicles  from  tech- 
nological w«r  plan — 
weighted  for  each 
operational  re- 
quirement 

Technologies  linked 
to  systems  by  ap- 
plicability factor; 
separate  technology 
goals 

Confidence  level 
gauge  of  achievement; 
contribution  value 
links  task  to  systems 
and  technological 
goals 

Funding  for  different 
levels  of  effort 

5.  Cornell 

Aeronautical 

Laboratory 

QMDO  description  of 
hardware.  Weighting 
proposed 

Combinations  of 
technologies  de- 
scribe possible 
technical  approaches 
to  subsystems 

Ordered  tasks  makeup 
technical  approach 
resource  pattern 

Direct  fund3  and 
other  resources 
needed 

6 . Hercules 
Corporation 

Projected  systems 
using  technology; 
contract  research — 
sales,  margins 

Substitute  and  com- 
plementary tech- 
nologies in  each 
system 

Technical  goals  of 
specific  organized 
units  of  effort 

Funding  for  each  pos- 
sible level  of  effort 

7.  Army  Missile 

Plan 

Systems  to  fulfill 
operational  require- 
ments—priorities 

I 1 

t 

Technical  composition 
of  system  and  subsys- 
tem concepts ; genera  1 
functional  capabili- 
ties 

Labs  plan  tasks  for 
technical  goals; 
tasks  rated  by  con- 
tribution 

Manpower,  funding  for 
planned  tasks 

8.  A.F.  Directorate 
of  Laboratories 
Plan 

System  concept  pos- 
sibilities— most 
probable  systems— 
subsystems 

Technical  planning 
objectives — technical 
gaps  for  systems; 
technical  goals 

Program  elements — 
projects — tasks — 
technical  efforts 

Summary  of  resources 
for  technical  ef- 
forts—program  ele- 
ments 

9.  Army  Research 

Plan 

Not  explicit— con- 
tained in  long  range 
technological  fore- 
cast 

j 

Potential  relevance 
of  technologies  in 

30  S. 2 elements  to 
each  OCO 

237  projects  in  6.2 
elements;  extra  Army 
effort 

Funding  trends  and 
planned  support  of 
elements — adequacy  of 
support 

10.  Another  Service 
Method 

1 

on# 

Technical  objectives 
related  to  opera- 
tional requirements 

7S0  tasks — weights 
reflect  contribution 
of  each  to  each  oper- 
ational requirement 

Estimated  funding  re- 
quired to  complete 
project 
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1 

Constraints 

Decision  Algorithm 

I Timing 

Risk 

Method 

Time-phasing  of  re- 
sources and  results 

Probability  of  actual 
outcomes  differing 
from  expected 

Limits  on  possible 
program  options 

Choosing  pattern  of 
development  work  to 
be  done 

1. 

Industrial 

Analog 

None 

None 

None 

Not  specific — DoD 
funding  proportional 
to  industry  in  some 
way 

2. 

TORQUE 

Weapon  ICC— tech- 
nical objectives  — 
funding  (timeliness 
function) 

None 

Total  budget 

iValue/aCost  in  de- 
scending order  among 
project  tasks 

3. 

Naval  Ordnance 
Laboratory 

Not  explicit— prog- 
ress possible  in 
next  budget  year 

. 

None 

Total  budget 

aPayoff/aCost  equated 
over  all  technologies 

4. 

A.F.  Flight 

Dynamics 

Laboratory 

Timing  of  technical 
objective,  weighted 
by  timeliness  func- 
tion 

) 

Probability  chat  task, 
output  will  fit  sys- 
tem 

Total  funds,  in- 
house  and  contract 
engineers 

Linear  program 

5. 

Cornell 
Aeronautlca  3. 
Laboratory 

Timing  of  QMDO'S 
tasks 

Probability  of  suc- 
cess for  each  task, 
technical  approach, 
subsystem 

QMDO  requirements, 
funds,  other  re- 
sources 

Decrease  budget  re- 
quired by  eliminating 
low  a "Exp.  Valua"/  ' 
aCosc  among  technical 
approaches 

6. 

Hercules 

Corporation 

Timeliness  of  market- 
ing and  time  required 
for  technical  goals 

Probability  of  pre- 
dicted sales,  mar- 
gins, research  needs, 
technical  timing,  and 
success 

Total  budget 

Search  possible 
shifts  of  funds  among 
projects  for  mix 
giving  maximum  pro- 
fits 

7. 

Army  Missile 

Plan 

Timing  of  funds, 
tasks,  target  dates 
of  systems 

None 

None 

Funds  distributed 
down  list  of  tasks 
ordered  by  priorities 

a. 

A.F.  Directorate 
of  Laboratories 
Plan 

Desired  capabilities 
set  out  in  five  year 
increments 

None 

Nona 

None 

9. 

Army  Research 

Plan 

Operational  require- 
ments out  to  20 
years;  development 
program  in  next  1-5 
years 

Implicit -- funding 
that  gives  reasonable 
probability  of  reach- 
ing technical  goal  on 
time 

In-house  capability, 
external  efforts, 
directed  work 

Qualitative  analysis 
of  shifts  in  emphasis 
warranted  from  prior- 
ities, relevance, 
support 

10. 

Another  Service 
Method 

Projects  set  up  to 
support  operational 
requirements  of  1982 

Probability  of  suc- 
cess of  each  task  to 
reach  technical  ob- 
jective with  funds, 
and  in  time 

Sponsored  projects, 
in-house  capability 

Rank  projects  by  de- 
scending exp.  value/ 
cost 
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a large  degree,  is  directly  correlated  with  the  magnitude  of 
the  resources  committed,  which  certainly  should  not  come  as 
any  surprise. 

D.  IN  RETROSPECT 

And  so  some  general  conclusions  and  statements  can  be 
gleaned  from  a review  of  these  predictive  models.  Firstly, 
the  worth-assessment  process  has  not  really  been  adequately 
structured  by  the  models  for  the  specific  goals  and  purposes 
of  the  military  R&D  system,  particularly  the  EDP.  To  a 
large  extent,  this  has  been  due  to  a generation  of  philoso- 
phies which  restrict  the  options  for  dealing  with  future 
world  situations  by  proposing  concepts  that  are  extensions  of 
the  current  force  structure  and  generally  use  the  current 
technological  approach  to  accomplish  mission  tasks.  Conse- 
quently, and  as  previously  indicated,  R&D  activities  tend  to 
focus  on  meeting  system  requirements  rather  than  on  gener- 
ating innovative  ways  to  accomplish  mission  tasks.  All  too 
often  the  R&D  activities  in  the  EDP  which  are  evaluated  are 
not  independent,  as  is  necessary  for  worth-assessment  proce- 
dures, but  are  often  interrelated  activities.  Consequently, 
the  projects  selected  usually  do  not  provide  all  the  technol- 
ogies necessary  to  develop  and  deploy  a desired  system. 

Secondly,  the  models  exercised  did  not  adequately  provide 
procedures  to  assure  the  inputs  necessary  to  meet  the 
assumptions  and  requirements  of  weighting  criteria.  That  is 
to  say,  missions,  goals,  and  operational  requirements  were  not 
(are  not)  being  defined  in  terms  that  are  worth- independent 
(Ref.  17)  . 
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Additionally,  different  worth  criteria  and  underlying 
assessment  processes  are  required  to  assess  different  activi- 
ties. Each  model  must  therefore  be  specifically  tailored  to 
a particular  activity.  Then  too,  the  cost  effectiveness  of 
actually  using  the  models  is  very  difficult  to  evaluate.  The 
true  effectiveness  should  generally  be  subjectively  evaluated 
in  terms  of  how  useful  an  R&D  manager  finds  the  results. 

The  future  of  R&D  planning  models  similar  to  those  identi- 
fied thus  far  is  not  particularly  promising  in  light  of  the 
observed  methodological  problems  and  managerial  resistance  to 
their  use.  The  models  just  cannot,  or  have  not  been  able  to 
capture  all  the  relevant  and  subtle  aspects  of  the  R&D  and  EDP 
planning  process.  The  methodological  problems  preclude  the 
use  of  such  models  even  as  decision  information  systems  to  aid 
managers,  since  their  output  is  highly  questionable. 

But  organizations  performing  Exploratory  Development  within 
R&D  must  still  select  and  evaluate  among  many  possible  activi- 
ties, however.  These  decisions  can  be  improved  by  a better 
understanding  of  both  the  process  of  innovation  and  the  Explor- 
atory Development  planning  process.  It  is  important  that 
organizations  have  systematic  procedures  and  logical  organi- 
zational structures  to  assure  that  the  major  planning  tasks  are 
effectively  accomplished.  Figure  6 has  been  suggested  as  a 
representative  outline  of  the  necessary  elements  of  a contempo- 
rary planning  process  (Ref.  17).  To  develop  such  procedures 
and  structures,  the  organization  must  clearly  define  what  it 
hopes  to  accomplish  by  performing  the  exploratory  development 
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IDENTIFICATION  AND  RANKING 
OF  OFERATIONAL  PROBLEMS 


LONG  RANGE 
OFERATIONAL 
GOALS 


OUTFLfT:  NEW  OFERATIONAL 
CAFABIUTIE8 , OBJECTIVES, 
NEEDS. 


CAPABILITIES 

OF 

PROGRAMMED 

FORCES 


EVALUATION 


EVALUATION  OF  EACH 
CANDIDATE  AS  A SOLUTION 
TO  IDENTIFIED  PROBLEMS 


FORECASTS 


OUTPUTS  LIST  OF  PROJECTS 
MATCHED  TO  RANKED 
OPERATIONAL  PROBLEMS 


MENU  OF  RED 
PROJECTS 


SELECTION  OF  PREFERRED 
PROJECTS  AND  ALLOCATION  OF 
FUNDS  AMONG  THOSE  PROJECTS 


ESTIMATED 

COSTS 


OUTPUTS  THE  RBO  BUDGET 
FOR  THE  NEXT  FERIOO 


DECISION 

CRITERIA 


B 

OTHER 

BUDGET 

Pigure  6 

CONSTRAINTS 
ANO  GOALS 

Conceptual  outline  of  a long  range  research 
development  planning  process. 
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within  R&D.  No  technique  , model  or  methodology  is  likely 
to  improve  the  effectiveness  of  a poorly  or  inadequately 
implemented  planning  process. 
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V.  DETERMINING  ROI  IN  THE  EXPLORATORY  DEVELOPMENT  PROGRAM 


It  is  difficult  to  argue  with  the  fundamental  proposition 
that  any  large  enterprise  ought  to  seek  evidence  of  the  effec- 
tiveness of  existing  programs  and  proposed  program  alterna- 
tives as  a basic  guide  to  decision  making.  Yet  few  sound 
evaluation  schemes  and  even  fewer  rigorous  field  experiments 
have  actually  been  undertaken.  Serious  questions  exist  con- 
cerning the  capacity  of  the  government  both  to  develop  and 
use  evaluative  results.  Much  of  the  concern  deals  with  the 
short  supply  of  competent  policy-oriented  researchers  both 
within  and  outside  of  the  government,  the  inadequacy  of  avail- 
able methods  and  concepts  for  carrying  out  evaluative  studies, 
and  the  shortage  of  organizations  with  the  capacity  to  under- 
take large-scale  evaluative  activities. 

An  evaluation  method  based  on  the  tools  of  social  research 
has  been  applied  successfully  to  social  action  programs  to 
determine  program  effectiveness.  The  basic  principles  of  this 
methodology,  while  developed  originally  for  social  and  educa- 
tion programs,  actually  are  oriented  toward  any  action  program 
which  can  attest  to  a sound  platform  of  goals  and  objectives. 
Thus,  there  appears  to  be  a potentially  viable  technique 
which  would  be  applicable  to  programs  exhibiting  outputs  which 
present  themselves  in  a somewhat  unquantifiable  manner. 

This  section  presents  an  overview  of  the  methodology, 
called  Evaluation  Research  CER) , and  attempts  to  show  how  it 
might  be  applied  to  programs  within  the  Exploratory  Development 
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Program.  While  a complete  treatise  and  development  of  an 
evaluation  study  on  an  exploratory  program  is  beyond  the 
scope  of  this  paper,  it  is  felt  that  the  ER  technique  could 
be  applied  in  particular  applications  to  yield  realistic 
measures  of  effectiveness  of  technology  programs.  A presen- 
tation of  the  background  for  the  basic  elements  of  such  a 
study  follows . 

A.  EVALUATION  RESEARCH  FOR  THE  EDP 

Evaluation  is  a rather  elastic  word  but  common  usage 
would  generally  imply  the  notion  of  judging  merit.  Evalua- 
tion has  frequently  had  explicitly  political  overtones.  It 
is  designed  to  yield  conclusions  about  the  worth  of  programs 
and,  in  so  doing,  is  intended  to  affect  the  allocation  of 
resources. 

A form  of  evaluation,  called  Evaluation  Research,  has 
been  devised  and  used  in  the  past  on  social  programs,  although 
in  its  strictest  sense  is  concerned  with  finding  out  how  well 
action  programs  work.  It  represents  the  application  of  social 
science  research  methods  to  discover  information  of  importance 
to  program  practice  and  public  policy.  The  tools  of  research 
are  applied  to  make  the  judging  process  more  accurate  and 
objective.  The  evaluation  thus  establishes  clear  and  specific 
criteria  for  success. 

The  research  process  is  particularly  important  when  (1) 
the  outcomes  to  be  evaluated  are  complex,  hard  to  observe, 
made  up  of  many  elements  reacting  in  diverse  ways;  (2)  the 
decisions  that  will  follow  are  important  and  expensive;  and 
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(3)  evidence  is  needed  to  convince  other  people  about  the 
validity  of  the  conclusions  (Ref.  18) . Clearly  then,  this 
scheme  appears  to  have  commonality  with  the  R&D  process  in 
general  and  the  Exploratory  Development  Program  in  particular. 

The  rationale  of  Evaluation  Research  is  that  it  provides 
evidence  on  which  to  base  decisions  about  maintaining, 
insituationalizing  and  expanding  successful  programs  and 
modifying  or  abandoning  unsuccessful  ones.  For  example,  the 
methodology  of  determining  program  selection  at  a laboratory 
Block  Program  Office  utilizes  a decision  matrix,  as  shown  in 
Table  V,  provides  much  of  the  necessary  basic  information 
for  initiating  an  evaluation  research  technique.  The  matrix 
is  utilized  as  a tool  for  determining  which  programs  should 
be  supported  and  continued,  as  they  relate  to  mission  needs. 
Used  in  this  context,  evaluation  research  is  increasingly 
important  as  a source  of  knowledge  and  direction.  It  can  be 
used  to  tell  which  programs  work  and  which  do  not,  and  points 
the  way  to  better  formulation  of  policy  and  program.  It  can 
be  even  more  helpful  as  it  proceeds  to  identify  the  effects 
of  specific  strategies  and  components  within  programs  and 
separates  out  those  that  contribute  to  favorable  outcomes 
from  those  that  are  ineffective  or  counter  productive  (Ref. 

181 . 

If  evaluation  and  cost-benefit  analysis  are  to  have  any 
effect  on  decisions,  there  has  to  be  some  way  to  move  from 
the  data  to  the  decision — some  system  for  transmitting 
information  and  feeding  it  into  the  decision  process.  The 
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Table  V.  Block  Program  Decision  Matrix 
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Planning,  Programming , Budgeting  System  (PPBS)  recently  in 
use  in  DoD  is  such  a system,  but  it  starts  with  the  definition 
of  objectives  rather  than  the  delineation  of  goals.  Providing 
a systematic  approach  to  governmental  planning,  the  distinctive 
characteristics  of  PPBS  ares 

1.  It  identifies  and  defines  fundamental  objectives  and 
considers  all  activities  in  terms  of  those  objectives. 

2.  It  explicitly  and  systematically  identifies  alterna- 
tive ways  of  carrying  out  the  objectives. 

3.  It  estimates  the  total  cost  implications  of  each 
alternative,  looking  not  only  at  the  current  year  but  also 
future  years. 

4.  It  estimates  the  expected  benefits  of  each  alternative. 

5.  It  presents  the  resulting  cost  and  benefit  comparisons 
for  each  alternative,  along  with  identification  of  major 
assumptions  and  uncertainties. 

PPBS  is  more  concerned  with  future  options  than  with  past 
experience,  especially  when  applied  in  the  technology  program 
area.  But  in  estimating  expected  benefits,  it  has  to  derive 
its  figures  from  somewhere,  and  this  is  where  evaluation 
should  come  in. 

B.  PURPOSE  OF  EVALUATION  RESEARCH 

Evaluation  research  is  viewed  by  many  as  a way  to  increase 
the  rationality  of  decision  making  and  policy  making.  With 
objective  information  on  the  outcomes  of  programs,  wise 
decisions  can  be  made  on  budget  allocations  and  program 
planning.  Programs  that  yield  good  results  will  be  expanded; 
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those  that  make  poor  showings  will  be  abandoned  or  drasti- 
cally modified. 

An  evaluation  study  does  not  generally  come  up  with  final 
and  unequivocal  findings  about  the  worth  of  a program.  Its 
results  often  show  small,  ambiguous  changes,  minor  effects, 
outcomes  influenced  by  the  specific  events  of  the  place  and 
the  moment.  It  may  require  continued  study  over  time  and 
across  projects  to  speak  with  confidence  about  success  and 
failure . 

For  decision  makers,  evaluation  evidence  of  outcome  is 
only  one  input  out  of  many.  They  must  consider  a host  of 
other  factors,  from  public  receptivity  to  participant  reac- 
tion, costs,  availability  of  staff  and  facilities,  and 
possible  alternatives.  Evaluation  will  definitely  not  take 
politics  out  of  the  decision  making  process. 

The  purpose  of  evaluation  research  is  to  measure  the 
effects  of  a program  against  the  goals  it  set  out  to  accom- 
plish as  a means  of  contributing  to  subsequent  decision 
making  about  the  program  and  improving  future  programming 
(Ref.  19) . The  measurement  of  the  effects  refers  to  util- 
izing a research  methodology  to  investigate  outcomes  of  the 
program  rather  than  efficiency  or  adherence  to  standards. 

The  comparison  of  effects  with  goals  stresses  the  use  of 
explicit  criteria  for  judging  how  well  the  program  is  doing. 

Evaluation  research  is  committed  to  the  principle  of 
utility.  If  it  is  not  going  to  have  any  effect  on  the 
decision,  then  it  is  clearly  an  exercise  in  futility.  Weiss 
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(Ref.  19)  suggests  that  there  are  probably  four  kinds  of 
circumstances  in  which  evaluation  is  probably  not  worth  doing: 

1.  When  there  are  no  questions  about  the  program.  It 
goes  on,  and  decisions  about  its  future  either  do 
not  come  up  or  have  already  been  made. 

2.  When  the  program  has  no  clear  orientation.  Program 
staff  improvise  activities  from  day  to  day,  based 
on  little  thought  and  less  principle,  and  the  pro- 
gram shifts  and  changes,  wanders  around  and  seeks 
direction. 

3.  When  people  who  should  know  cannot  agree  on  what  the 
program  is  trying  to  achieve.  If  there  are  vast 
discrepancies  in  perceived  goals,  evaluation  has  no 
ground  to  stand  on. 

4.  When  there  is  not  enough  money  or  staff  sufficiently 
qualified  to  conduct  the  evaluation.  Evaluation  is 
a demanding  business,  requiring  time,  money, 
imagination,  tenacity  and  skill. 

The  all-purpose  evaluation  is  a myth.  But  there  are  several 
types  of  uses  for  evaluation.  Evaluation  can  investigate  the 
extent  of  program  success  so  that  decisions  such  as  these  can 
be  made: 

1.  To  continue  or  discontinue  the  program. 

2.  To  improve  its  practices  or  procedures. 

3.  To  add  or  drop  specific  program  strategies  and 
techniques. 

4.  To  institute  similar  program  elsewhere. 

5.  To  allocate  resources  among  competing  programs. 

6.  To  accept  or  reject  a program  approach  or  theory. 


C.  ELEMENTS  OF  AN  EVALUATION  RESEARCH  PROCESS 


In  abstract,  the  basic  elements  of  the  evaluation  research 
process  include  (1)  finding  out  the  program's  goals,  (2)  trans- 
lating the  goals  into  measurable  indicators  of  goal  achievement. 
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(3)  collecting  data  on  the  indicators  of  the  participating 
elements  (and  for  an  equivalent  control  group) , and  (4) 
comparing  the  data  on  participants  (and  control  elements) 
with  the  goal  criteria. 

But  this  process  in  theory  becomes  much  more  complex  in 
practice.  A host  of  unexpected  problems  invariably  arise 
such  as : 

1.  Program  goals  are  often  hazy,  ambiguous,  hard  to  pin 
down. 

2.  Programs  not  only  move  toward  official  goals,  but  they 
also  accomplish  other  things,  sometimes  in  addition  and  some- 
times instead. 

3 . The  program  is  a congeries  of  activities , people  and 
structures,  some  irrevelant. 

4.  The  evaluation  question  as  posed  ignores  the  issue  of 
why  the  program  succeeds  or  fails. 

Ideally  then  Weiss  (Ref.  18)  suggests  the  folowing  as 
the  necessary  core  issues  to  comprise  the  effective  evaluation 
research  process: 

1.  Formulate  the  program  goals  that  the  evaluation  will 
use  as  criteria. 

2.  Choose  among  multiple  goals. 

3.  Investigate  unanticipated  consequences. 

4 . Measure  outcomes . 

5.  Specify  what  the  program  is. 

6.  Measure  program  inputs  and  intervening  processes. 

7.  Collect  and  compare  necessary  data. 
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In  its  rudimentary  form  then,  the  evaluation  process 
encompasses  the  interpretation  and  implementation,  including 
the  elements  of  policy,  goals,  theory,  models,  experiment 
(to  check  theory  and  provide  criteria  for  success-failure) . 

The  experimental  model  remains  the  ideal  in  evaluation 
methodology,  with  random  assignment  of  subjects  to  an  experi- 
mental group  which  is  exposed  to  the  program  stimulus  and  to 
a control  group  which  is  not,  thus  providing  the  research 
rationale. 

It  collects  before  and  after  measures  over  a full  cycle 
of  the  program  and  uses  specified  goal  criteria.  Its  results, 
then,  disclose  the  extent  to  which  a consistent  program  has 
reached  its  stated  goals.  However,  it  rarely  distinguishes 
why  the  observed  results  occur,  what  processes  intervene 
between  input  and  outcome  or  what  the  particular  implications 
are  for  improving  the  effectiveness  of  the  program. 

The  application  to  the  EDP,  however,  might  just  provide 
the  necessary  means  with  which  to  conduct  a satisfactory 
selection/evaluation  process,  ultimately  yielding  a rational 
method  for  determining  program  worth.  The  stage  is  set;  what 
remains  now  is  a progressive-oriented  faction  of  the  Explora- 
tory Development  Programs , willing  to  commit  the  necessary 
resources  of  time,  personnel  and  money  to  explore  the  poten- 
tial of  this  new  technique. 
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VI.  SUMMARY  AND  CONCLUSIONS 


The  Navy  Exploratory  Development  Program  is  currently 
undergoing  major  revision  to  align  its  management  planning 
system  with  the  philosophies  of  the  new  wave  of  thinking 
regarding  program  effectiveness;  i.e.,  to  meet  the  Navy's 
basic  obligation  to  put  government  resources  to  their  most 
effective  use  in  mission  accomplishment. 

Traditionally,  technology  has  gone  on  to  develop  subsystems 
and  systems  concepts  independently  of  specific  mission  needs. 
Further,  rather  than  actively  building  a strong  technology 
base  through  innovation,  old  technologies  have  been  extended 
to  improve  performance  of  old  kinds  of  products.  New  systems, 
while  becoming  more  complex  and  expensive  have  actually  become 
less  technologically  advanced. 

It  is  recognized  that  it  is  imperative  that  cost-benefits 
be  maximized  on  technology  base  programs.  This,  though,  can 
only  be  realized  when  the  base  programs  are  driven  from  oper- 
ational needs  and  mission  requirements. 

A complete  and  sound  understanding  of  the  functions  and 
underlying  philosophies  and  policies  of  the  technology  pro- 
gram is  essential  to  determine  that  program's  worth,  or 
return-on- inves  tment . 

Sound  foundations  must  be  established  from  which  goals  and 
strategies  can  be  derived  to  provide  meaningful  inputs  for 
evaluation  methodologies  of  contemporary  exploitation,  rather 
than  basing  output  measures  on  dollar  values. 
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In  terms  of  the  policy  platform  for  the  Exploratory  Devel- 
opment Program,  the  underlying  principles  evidenced  in  the 
new  rationale  include: 

• Supportive  of  technological  advances  being  of  funda- 
mental importance  to  operational  needs . 

• Quest  for  technical  and  managerial  excellence  at  all 
levels . 

• Recognition  of  the  necessity  for  both  top-down  direction 
and  bottom-up  inputs . 

• Accountability  essential  at  all  levels. 

• Necessity  for  strategies  as  tools  to  implement  policies. 

• Recognition  that  sometimes  technical  advance  must  be 
injected  into  the  main  stream  of  the  system  forcibly. 

• Recognition  of  essential  need  for  adequate  information 
systems  to  allow  interaction. 

With  a strong  push  in  Navy  policy  to  promote  accountability , 
a more  decentralized  organization  is  developing,  as  evidenced 
by  the  formation  of  the  Block  Program  Offices.  This  allows 
a means  for  providing  technical  management  in  the  laboratories 
and  Systems  Commands  with  the  necessary  authority  to  plan  and 
manage  technical  efforts  related  to  the  pursuit  of  significant 
objectives. 

As  a result  of  direction  by  the  ASN  (R&D)  to  establish  and 
implement  a program  to  improve  the  exploratory  development  sys- 
tem, to  improve  higher  echelons'  understanding  of  the  system 
and  to  justify  6.2  expenditures,  the  Exploratory  Development 
Program  has  been,  and  currently  is  being  radically  revamped. 
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• Provide  top-down  direction  by  the  use  of  technical 
strategies,  prioritized  technical  objectives  and 
definition  of  major  program  thrusts  designed  to  meet 
operational  requirements. 

• Provide  visibility  by  relating  the  program  to  Navy 
functions  and  operational  needs. 

• Promote  innovation  and  concentration  on  highest  leverage 
opportunities . 

• Expedite  transfer  of  projects  to  advanced  development 
(6.3)  and  use  of  new  technical  capabilities  developed 
in  research  (6.1). 

• Demonstrate  program  relevance  and  cost-effectiveness. 

It  is  essential  that  a firm  understanding  of  the  technical 
strategies , functional  areas  and  program  elements  be  had  to 
augment  an  understanding  of  the  EDP  objectives  and  transition 
steps.  Only  through  a thorough  knowledge  of  the  inner  work- 
ings of  the  new  system  can  any  meaningful  review,  evaluation 
and  assessment  methodology  be  realized.  While  no  established 
program  exists  currently,  there  certainly  is  potential  for 
developing  a viable  methodology,  particularly  with  the  well 
structured  and  goal  oriented  EDP  being  implemented  today. 

Worth-assessment  processes  to  date  have  not  been  able  to 
be  adequately  applied  to  the  R&D  structure  in  general,  nor  the 
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EDP  in  particular.  This  is  due  primarily  to  the  traditional 
programs  being  system  requirement  rather  than  mission  need 
(goal)  oriented.  Further,  earlier  models  did  not  define  in- 
puts in  terms  that  were  worth-dependent  because  different 
worth  criteria  and  underlying  assessment  processes  must  be 
specifically  tailored  to  particular  activities. 

The  future  of  R&D  planning  models  is  not  particularly 
promising  in  light  of  observed  methodological  problems  and 
managerial  resistance  to  their  use.  The  models  just  have  not 
been  able  to  capture  all  of  the  relevant  and  subtle  aspects 
of  the  R&D  and  the  EDP  planning  process. 

But  organizations  performing  exploratory  development  must 
still  select  and  evaluate  programs  some  way.  These  decisions 
can  be  improved  through  a better  understanding  of  both  the 
process  of  innovation  and  the  exploratory  development  planning 
process . 

New  techniques  and  methodologies,  as  well  as  revisions  of 
old  ones,  are  constantly  being  identified,  many  receiving  par- 
ticular area  utilization,  which  will  contribute  to  the  process 
of  rational  decision  making;  i.e. , program  evaluation  and 
assessment.  One  method  felt  to  have  high  potential  in  the 
exploratory  development  area  is  that  called  Evaluation  Research. 
Originally  developed  for  application  to  social  action  programs, 
it  is  best  utilized  in  situations  where  sound  platforms  of 
goals  and  objectives  can  be  established.  Thus,  there  appears 
to  be  a potentially  viable  technique  which  might  be  applied  to 
programs  in  the  EPD  which  yield  outputs  in  a somewhat  unquanti- 
fiable  manner. 
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The  stage  has  been  set  for  the  design  of  a new  methodo- 
logical tool  to  assess  the  worth  of  programs  in  the  EDP. 

A technique  has  been  identified  which  has  high  potential.  It 
is  strongly  recommended  that  further  study  be  undertaken, 
using  the  background  and  management  planning  systems  presented 
in  this  thesis,  to  ultimately  demonstrate  the  ability  of  eval- 
uation research  to  effectively  determine  a measure  of  worth 
of  programs  in  the  Exploratory  Development  Program. 

The  evidence  suggests  that  such  a tool  can  be  realized. 
What  is  required  now  is  to  uncover  a progressively  oriented 
faction  of  the  EDP  willing  to  commit  the  necessary  resources 
to  more  fully  explore  and  demonstrate  the  potential  of  this 
research  technique. 
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APPENDIX  A 


EXPLORATORY  DEVELOPMENT  PROGRAM  RESPONSIBILITIES 

The  following  delineates  Exploratory  Development  Program 
management  responsibilities.  It  discusses  basic  organization 
design  principles,  the  responsiblities  of  Program  organiza- 
tions and  officials,  and  the  various  roles  played  by  non- 
NAVMAT  organizations  and  officials  in  the  6.2  Program. 

Organization  Design  Concepts.  The  Exploratory  Development 
Program  management  system  is  designed  to  strike  the  best  com- 
promise between  the  need  to  administer  public  resources  in 
accordance  with  applicable  laws,  regulations,  and  policies, 
and  the  need  to  provide  technical  management  with  the  freedom 
required  to  hold  them  accountable  for  results.  The  responsi- 
bilities described  for  the  Exploratory  Development  Program  are 
based  on  the  following  design  principles: 

Fixed  accountability.  Establishing  the  conditions  neces- 
ary  to  hold  individual  officials  (as  distinguished  from  diffuse 
bureaucracies)  accountable  is  a central  consideration.  Pre- 
requisites to  accountability  include: 

a.  Assigning  responsibility  for  results  to  an  individual. 

b.  Provide  that  individual  with  the  necessary  authority 
and  resources  to  accomplish  the  required  results.  If  a mana- 
ger is  to  be  held  accountable,  he  must  have  the  opportunity 
to  pursue  objectives  by  the  means  which  he  judges  most 
effective. 
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c.  Providing  the  responsible  individual  with  the  oppor- 
tunity to  perform  by  limiting  the  demands  on  his  time.  This 
includes  demands  by  superiors  and  their  staffs , and  by  other 
external  officials  and  organizations. 

Clear  separation  of  responsibilities.  Responsibilities 
for  determing  the  objectives  to  be  pursued,  and  the  technical 
effort  required  to  fulfill  those  objectives,  have  been  clearly 
and  deliberately  separated.  Although  it  is  essential  that 
spokesmen  for  these  two  interests  conduct  an  in-depth  dia- 
logue in  the  process  of  defining  the  Program,  it  is  important 
to  avoid  conflict  of  interest.  In  the  management  system 
described  herein,  technical  people  are  not  in  a position  to 
dictate  "requirements , " and  management  people  and  sponsors 
are  not  in  a position  to  control  the  selection  of  technical 
approach. 

Quality  information  for  management.  The  management  sys- 
tem has  been  designed  to  provide  the  ASN  (R&D) , CND,  and 
lower-level  officials  with  all  the  information  they  require 
to  implement  their  Program  responsibilities.  The  interaction 
of  organizations  generates  quantitative  information  required 
for  technical  management  to  put  technical  resources  to  their 
most  productive  use,  and  for  higher-level  management  to 
reallocate  resources  as  required  to  increase  overall  Program 
output.  A system  of  institutionalized  Program  Reviews  ensures 
the  quality  of  information  provided  to  top  management. 
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Management  by  objectives  and  results.  The  management 
system  relies  on  mutually  established  and  agreed-upon  objec- 
tives and  goals  rather  than  on  management  by  specific  direc- 
tives, detailed  procedures,  elaborate  reporting  systems, 
frequent  reviews,  and  other  close  controls.  Management  is 
by  "self-control"  in  pursuit  of  the  most  efficient  accomplish- 
ments of  stated  objectives.  Higher-level  review  primarily 
measures  results  achieved  in  relation  to  approved  objectives. 

"People  information  system"  for  coupling.  The  process  of 
defining  EDP  projects  which  collectively  support  Department 
of  the  Navy  purposes  requires  extensive  and  intimate  inter- 
action between  competent  spokesmen  for  both  sponsor  and  per- 
former organizations.  The  interaction  of  Technical  Strategists 
with  spokesmen  for  Mission  Needs  and  with  technical  managers 
in  the  laboratories  is  a vital  element  of  the  "people  infor- 
mation system."  This  face-to-face,  paper-free  information 
system  provides  a mechanism  for  the  flow  of  information  on 
needs  and  operational  environment  to  the  technical  community, 
and  for  the  reverse  flow  of  knowledge  about  technical  oppor- 
tunity to  the  CNO/CMC  staffs. 

Program  Management  Responsibilities 

Deputy  Chief  of  Naval  Material  (Development)  ( DNM ( D ) / 

Chief  of  Naval  Development  (CND) ) . As  DCNM(D) , this  officer 
reports  directly  to  CNM.  As  CND,  he  is  directly  responsible 
to  ASN  (R&D) . DCNM(D)/CND  has  overall  control  of  the  Explor- 
atory Development  Program  and  its  funds.  He  carries  out  this 
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general  responsibility  primarily  through  the  DNT.  His  func- 
tions include: 


a.  Reviewing  and  approving  the  Exploratory  Development 
Program  Plan  (EDPP)  at  each  stage  in  its  evolution  (i.e.,  the 
Draft  and  Tentative  EDPPs,  as  well  as  the  EDPP. 

b.  Appointing  Technical  Strategists. 

c.  Approving  Technical  Strategies. 

d.  Approving  Program  Plans  and  related  budgets. 

e.  Directing  the  allocation  of  funds. 


Director  of  Technology  Programs /Director  of  Navy  Technol- 
ogy DTP/DNT.  As  DNT,  this  official  is  directly  responsible 
to  the  ASN  (R&D) . As  DTP,  he  is  responsible  to  the  DCNM(D) / 

CND.  Acting  for  the  CND,  he  is  responsible  for  the  planning, 
execution,  and  evaluation  of  the  Exploratory  Development  Pro- 
gram. His  functions  include  the  following: 

a.  Designing  a balanced  and  integrated  overall  Explora- 
tory Development  Program  which  maximizes  the  contribution  of 
the  Technology  Base  to  the  mission  capability  of  Department 
of  the  Navy  operating  forces . 

b.  Developing  and  documenting  the  Exploratory  Development 
Program  Plan  (EDPP)  for  review  and  approval  by  CND  and  ASN  (R&D) . 

c.  Nominating  Technical  Strategists  and  appointing 
Strategy  Team  members. 

d.  Reviewing  and  coordinating  Technical  Strategies. 

e.  Developing  a Program-wide  priority  list  for  concept- 
related  technical  effort. 
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f.  Negotiating  Program  Plans  (and  related  budgets)  with 
Systems  Command  Directors  of  Research  and  Technology  (SYSCOM 
03s)  and  with  Technical  Directors  of  the  laboratories  (Lab 
TDs) , and  recommending  those  plans  to  CND  for  approval. 

g.  Appointing  Program  Review  Team  leaders,  and  assisting 
in  the  staffing  of  Review  Teams. 

h.  Scheduling  and  chartering  External  Program  Review. 

i.  Conducting  Program  Reviews  at  the  SYSCOM  03/Lab  TD 
level. 

SYSCOM  Directors  of  Research  and  Technology  (SYSCOM  03s) . 
The  SYSCOM  0 3s  plan  and  execute  that  portion  of  the  overall 
Exploratory  Development  Program  related  to  System  Concepts 
(SCs)  and  other  products  within  the  material  responsibility 
areas  of  their  SYSCOM.  The  general  responsibilities  of  a 
SYSCOM  03  include: 

a.  Designing  a balanced  and  integrated  overall  SYSCOM 
Exploratory  Development  Program  which  supports  Objectives  in 
the  approved  Mission  Area  Technical  Strategies. 

b.  Documenting  planned  technical  efforts  in  a Program 
Plan  for  negotiation  with  and  approval  by  the  DNT . 

c.  Prioritizing  concept-related  objectives  set  forth  in 
the  Program  Plan. 

d.  Providing  designated  personnel  to  serve  as  Technical 
Strategists  and  as  members  of  Strategy  and  Program  Review 
Teams,  as  requested  by  DNT. 

e.  Negotiating  and  approving  proposed  TAPs. 
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f.  Reviewing  proposed  changes  in  TAPs  approving  desired 
changes  within  his  authority,  and  submitting  desired  changes 
beyond  his  authority  to  DNT  for  appropriate  action. 

g.  Monitoring  execution  of  his  SYSCOM's  technical  program, 
and  taking  actions  as  appropriate. 

h.  Ensuring  the  Internal  Review  of  his  SYSCOM's  program, 
and  providing  copies  of  Internal  Review  Reports  to  DNT. 

Laboratory  Technical  Directors  (Lab  TPs) . The  Lab  TDs  plan 
and  execute  that  portion  of  the  overall  Exploratory  Develop- 
ment Program  related  to  the  development  and  utilization  of 
technology  within  the  areas  of  Laboratory  responsibilities. 

The  general  responsibilities  of  the  Lab  TD  include  the  follow- 
ing functions: 

a.  Designing  a balanced  and  integrated  overall  Laboratory 
Exploratory  Development  Program  which  supports  approved  Tech- 
nology Area  Strategies. 

b.  Documenting  planned  technical  efforts  in  a Program 
Plan  for  negotiation  with  and  approval  by  the  DNT. 

c.  Prioritizing  technology-related  objectives  set  forth 
in  the  Program  Plan. 

d.  Providing  designated  personnel  to  serve  as  Technical 
Strategists  and  as  members  of  Strategy  and  Program  Review 
Teauns,  as  requested  by  DNT. 

e.  Negotiating  amd  approving  proposed  TAPs. 

f.  Monitoring  execution  of  his  Laboratory's  Technical 
Program,  and  taking  actions  as  appropriate. 
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g.  Reviewing  changes  in  proposed  TAPs  approving  desired 
changes  within  his  authority,  and  submitting  desired  changes 
beyond  his  authority  to  the  DNT  or  SYSCOM  03  as  appropriate. 

h.  Ensuring  the  Internal  Review  of  his  Laboratory's  pro- 
gram, and  providing  copies  of  Internal  Review  Reports  to  DNT. 

Responsibilities  for  preparation  of  Technical  Strategies. 
Each  Technical  Strategy  is  developed  and/or  updated  by  a Tech- 
nical Strategist  and  a supporting  Technical  Strategy  Team. 

Technical  Strategists.  A Technical  Strategist  is  respon- 
sible for  the  development  and  updating  of  each  Technical 
Strategy.  Strategists  report  to  DNT  for  all  matters  relating 
to  their  Technical  Strategies.  Technical  Strategists  may  be 
recruited  from  any  Department  of  the  Navy  organization; 
normally,  however,  they  will  be  NAVMAT , Laboratory,  or  Systems 
Command  officials  "double  hatted"  to  CND  for  this  duty.  The 
general  responsibilities  of  Technical  Strategists  include; 

a.  Identifying  Strategy  Team  needs  with  the  appropriate 
knowledge  and  organizational  affiliations. 

b.  Developing  and/or  updating  the  assigned  Technical 
Strategy  as  required  (with  the  aid  of  the  Strategy  Team) . 

c.  Maintaining  current  knowledge  of  the  Technical  Strategy 
in  general  (and  its  MA  or  TA  Objectives  in  particular)  and  of 
all  relevant  technical  capabilities , current  efforts,  and 
planned  efforts. 

d.  Reviewing  all  TAPs  which  purport  to  support  Objectives 
listed  in  the  Strategy,  and  taking  appropriate  actions  to  im- 
prove the  effectiveness  of  technical  contributions  to  higher- 
level  objectives. 
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e.  Serving  as  person-to-person  channels  between  CND/DNT, 
Strategy  Teams,  and  the  technical  community  for  the  flow  of 
needs  information  related  to  their  Technical  Strategies. 

This  function  includes  helping  technical  personnel  establish 
direct  contact  with  individuals  most  knowledgeable  about  issues 
of  interest. 

Strategy  Team.  Strategy  Teams  are  constituted  so  that 
they  have  within  the  Team  the  essential  knowledge  and  expertise 
required  for  the  development  of  the  Technical  Strategy — without 
reference  to  documents  except  for  detailed  information.  Such 
representation,  in  addition  to  helping  in  the  cost-effective 
preparation  of  an  adequate  Strategy  document,  is  an  essential 
element  in  the  development  and  maintenance  of  the  "people 
information  system."  Ideally,  the  Strategy  Team  includes 
members  from  organizations  which  both  require  information  on 
the  Strategy,  and  which  are  sources  of  information  required  in 
the  development  of  the  Strategy.  Each  Strategy  Team  should 
include  individuals  who  served  on  the  Team  in  prior  years; 
this  ensures  the  preservation  of  a paper-free  institutional 
memory.  MA  Strategy  Teams  include  members  from  all  TA  Strategy 
Teams  for  technologies  critical  to  the  attainment  of  the  MA 
Objectives.  Participation  of  these  people  provides  a direct 
and  paper-free  injection  of  information  on  technological  capa- 
bilities and  DOtential  capabilities  into  the  development  and 
updating  of  the  Strategy,  and  a reverse  flow  of  Mission  Need 
information  into  the  program  planning  and  execution  activities 
;f  their  parent  Systems  Commands  and  Laboratories . TA  Strategy 
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Teams  include  spokesmen  for  SCs  which  use  subject  technology, 
members  from  organizations  developing  subject  technology,  and 
representatives  of  the  research  community  involved  in  develop- 
ing the  base  of  knowledge  underlying  the  subject  technology. 

Source  of  Strategy  Team  members.  Strategy  Team  members 
are  recruited  from  both  Government  and  the  private  sector. 
Department  of  the  Navy  technical  organizations  are  directed 
to  provide  members  to  Technical  Strategists  on  a negotiated 
basis.  Assistance  in  recruiting  non-Department  of  the  Navy 
personnel  will  be  provided  by  DNT  (Attention  MAT  03T3) . 
Strategy  Team  members  from  Department  of  the  Navy  organiza- 
tions are  to  be  provided  travel  and  analytical  support  by 
their  parent  organizations  as  required. 

Program  Review  Teams.  There  are  two  kinds  of  Program 
Review:  Internal  Review  and  External  Review  (see  Chap.  IV) . 

Internal  Review  Teams,  under  the  leadership  of  the  cognizant 
Laboratory  Technical  Director  or  SYSCOM  03,  conduct  self- 
reviews of  all  detailed  technical  programs.  External  Review 
Teams,  under  the  leadership  of  a designated  member  of  DNT 1 s 
staff,  conduct  External  Program  Reviews  at  the  Laboratory/ 
SYSCOM  level. 

Staffing  the  Interval  Reviews.  SYSCOM  03s  and  Laboratory 
Technical  Directors  are  responsible  for  determining  the  size 
and  make-up  of  each  Laboratory  or  SYSCOM  Internal  Review 
Team,  and  for  assigning  individuals  from  within  their  respec- 
tive organizations  to  serve  as  Team  members.  TD/SYSCOM  03s 
also  have  the  option  of  recruiting  Team  members  from  outside 
their  organizations. 
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Staffing  External  Reviews.  Two  primary  considerations 
determine  the  staffing  of  External  Review  Teams : (1)  Team 

members  should  be  qualified  to  verify  the  validity  of  Internal 
Review  information  without  reference  to  additional  documenta- 
tion, and  (2)  the  External  Review  Process  should  establish 
a "people  information  system"  which  contributes  to  the  effec- 
tive planning  and  execution  of  the  overall  Program.  The  DNT- 
chartered  External  Program  Review  Team  is  a broad-based  ad  hoc 
group.  DNT  appoints  an  External  Review  Team  leader  from  his 
staff;  he  also  appoints  other  Team  members  from  among  inter- 
ested and  qualified  Navy  and  non-Navy  organizations . External 
Review  Team  members  include:  (1)  current  and/or  former  members 
of  relevant  Strategy  Teams,  (2)  DNT  staff  members,  (3)  members 
of  previous  DNT  Program  Review  Teams  (to  ensure  the  preserva- 
tion of  a paper-free  institutional  memory;  these  individuals 
can  note  what  has  been  done  about  deficiencies  discovered  in 
previous  External  Program  Reviews) , (4)  representatives  of 
organizations  which  supply  inputs  to  the  programs  under  review 
(e.g.,  ONR  Scientific  Officers  for  disciplines  critical  to  the 
technologies  involved) , (5)  representatives  of  organizations 
which  use  the  outputs  of  the  programs  under  review,  (6)  tech- 
nology managers  who  conduct  similar  programs,  and  (7)  experts 
from  the  wider  technical  community,  including  industry.  Extern- 
nal  Program  Review  Team  members  are  important  links  in  the 
paper-free  "people  information  system." 

DNT  Staff.  The  DNT  staff  provides  the  DNT  with  supporting 
services,  information,  and  counsel  required  for  effective 
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performance  of  DNT's  Exploratory  Development  Program  manage- 
ment functions.  Individual  staff  members  are  assigned  cogni- 
zance over  specific  matters — such  as  individual  Technical 
Strategies,  Laboratories  and  SYSCOMs  and  their  related  Pro- 
gram Plans,  and  Program  Elements.  Specific  duties  of  these 
cognizant  staff  members  are  discussed  below. 

Technical  Strategies.  Duties  of  the  DNT  staff  members 
assigned  cognizance  over  a Technical  Strategy  include: 

a.  Achieving  and  maintaining  in-depth  knowledge  of  the 
Technical  Strategy,  its  rationale,  and  of  all  on-going  or 
planned  technical  effort  supporting  Objectives  in  the  approved 
Strategy . 

b.  Serving  as  a member  of  the  Strategy  Team  for  that 
Strategy. 

c.  Providing  staff  services  for  the  recruitment  and 
assignment  of  Strategy  Team  members,  in  compliance  with  staffing 
requirements  developed  by  the  Technical  Strategist. 

d.  Alerting  the  DNT  to  developments  or  trends  related  to 
the  Strategy  which  require  DNT's  attention. 

Organization  and  Program  Plans.  Duties  of  DNT  staff  mem- 
bers assigned  cognizance  over  technical  organizations  and 
related  Program  Plans  include: 

a.  Achieving  and  maintaining  in-depth  knowledge  of  the 
planning  and  execution  aspects  of  the  organization's  technical 
program— including  all  changes  to  the  originally  approved 
program. 
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b.  Reviewing  proposed  Program  Plans  and  providing  DNT 
with  appropriate  information  and  counsel. 

c.  Serving  on  the  Program  Review  Team  (as  leader  or 
member) . 

d.  Determining  requirements  for  Review  Team  personnel, 
and  taking  necessary  actions  to  facilitate  the  recruitment 
and  assignment  of  appropriate  personnel. 

e.  Alerting  the  DNT  to  developments  or  trends  related 
to  the  organization  (or  its  Technical  program)  which  require 
DNT ' s attention . 

f.  Providing  oral  and  written  information  about  the 
organization  and  its  program  to  Government  officials  and  pri- 
vate sector  individuals  with  a need  to  know. 

Program  Elements.  Duties  of  DNT  staff  members  assigned 
cognizance  over  a Program  Element  include: 

a.  Achieving  and  maintaining  in-depth  knowledge  of: 

(1)  planning  and  execution  aspects  of  work  included  within  the 
Program  Element,  (.2)  technical  objectives  and  supporting 
rationales  for  the  work  performed,  and  (3)  all  actions  and 
policies  of  higher  level  officials,  including  the  Congress, 
related  to  the  Program  Element. 

b.  Preparing  Program  Element  Descriptions  (PEDs)  and  any 
other  documentation  required  in  connection  with  the  annual 
budget  submission,  apportionment  process.  Congressional 
hearings,  etc. 
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c.  Preparing  PE- level  TAP  Aggregates  using  TAP  informa- 
tion approved  by  SYSCOM  03s  and  Lab  TDs  (and  by  DC/S  RD&S  and 


CG  MCDEC  for  Marine  Corps  programs) . 

d.  Alerting  DNT  to  developments  or  trends  related  to  the 
Program  Element  which  require  DNT's  attention. 

e.  Providing  information,  both  orally  and  in  writing, 
concerning  the  Program  Element  to  Government  officials  and 
private  sector  individuals  with  a need  to  know. 


Interfacing  Responsibilities.  Other  organizations  and 
individuals  outside  of  the  Naval  Material  Command  play  signifi- 
cant roles  in  the  Exploratory  Development  Program. 

Congress.  The  Congress  provides  the  funds  for  the  6.2 
Program  and  establishes  funding  controls  at  the  Program  Ele- 
ment level.  Congress  may,  on  occasion,  control  the  funding  of 
individual  projects.  Management  responsiveness  to  Congress, 
credibility  of  information,  and  utility  of  results  are  essen- 
tial to  continued  stability  of  the  Navy's  Program. 

Director  of  Defense  Research  and  Engineering  (DDR&E) . The 
DDR&E  exerts  influence  at  all  levels  of  the  Program.  He  can 
compare  the  management  and  technical  content  of  the  Exploratory 
Development  Programs  of  all  Services,  and  can  direct  all  tech- 
nical programs  in  an  area  to  be  assigned  to  one  Service.  The 
DDRfcE  is  the  dominant  DoD  agency  in  the  justification  of  work 
and  the  budgeting  of  funds . 

Assistant  Secretary  of  the  Navy  (Research  and  Development 
(ASN (RfcD) ) . The  ASN  (R&D)  is  responsible  for  the  RDT&E (N) 
appropriation,  and  for  the  planning  and  execution  of  the  entire 
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Department  of  the  Navy  RDT&E  Program  (including  Exploratory 
Development) . ASN  (R&D)  reviews  and  approves  the  various 
versions  of  the  EDPP  on  an  exception  basis.  He  is  also  charged 
with  justification  of  the  Navy  EDP  to  DDR&E  and  Congress.  The 
small  ASN  (R&D)  staff  relies  heavily  on  the  CND  and  ms  staff 
for  information  and  presentation  of  the  program.  Close  work- 
ing relationships  and  mutual  confidence  between  CND/DNT  and 
the  ASN  (R&D)  are  essential. 

Chief  of  Naval  Operations /Commandant  of  the  Marine  Corps 
( CNO/CMC) . The  CNO  and  CMC  influence  the  6.2  Program  by  estab- 
lishing operational  needs  and  by  directly  sponsoring  Advanced 
and  Engineering  Development.  As  the  primary  markets  for  the 
Program  output,  the  CNO/CMC  must  be  satisfied  with  its  mili- 
tary relevance.  The  CND,  assisted,  by  the  SYSCOMs , interprets 
the  operational  needs  described  by  CNO/CMC  in  the  Technical 
Strategy  documents.  The  subsequent  discussion,  understanding, 
refinement,  and  definition  of  operational  needs  lead  to  devel- 
opment and  acceptance  of  a useful  product.  CNO/CMC  review 
all  SCs  when  they  are  first  submitted,  and  annually  as  long 
as  they  are  included  in  the  Program.  CNO/CMC  estimate  the 
military  worths  of  all  SCs  and  assign  them  quantitative  mili- 
tary worth  ratings,  "v"  ratings,  for  use  in  planning  Technical 
Strategies  and  supporting  technical  programs. 

Chief  of  Naval  Research  (CNR) . In  addition  to  his  respon- 
sibilities for  the  Naval  Research  Program,  the  CNR  is  responsi- 
ble for  sponsoring  these  exploratory  development  projects 
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which  are  necessary  to  ensure  orderly  transition  from  research 
to  development.  For  the  projects  which  he  sponsors,  CNR  func- 
tions like  a Systems  Command. 

Chief  of  Naval  Personnel  (CNP)  and  Chief,  Bureau  of 
Medicine  and  Surgery  (CHBUMED) . CNP/CHBUMED  are  the  primary 
sponsors  for  Exploratory  Development  relative  to  the  human 
subsystem  and  to  medical  and  dental  systems.  They  are  under- 
stood throughout  this  instruction  to  function  like  a Systems 
Command  in  these  areas. 

Indus try /Academia.  Industry  and  educational  institutions 
interact  principally  at  the  SYSCOM  and  Laboratory  levels. 

They  provide  Laboratories  and  SYSCOMs  with  assistance,  on  a 
contract  basis,  in  developing  both  technologies  and  products. 


APPENDIX  B 


EXPLORATORY  DEVELOPMENT  PROGRAM  DOCUMENTATION 

The  documentation  for  planning  and  justifying  the  Explora- 
tory Development  Program  is  described  in  this  appendix.  The 
basic  document  through  which  the  Chief  of  Naval  Development 
(CND)  and  the  Director  of  Navy  Technology  (DNT)  control  the 
Program  is  the  Exploratory  Development  Program  Plan  (EDPP) . 

The  EDPP  is  supported  by  Technical  Strategy  documents,  Labor- 
atory and  System  Command  Program  Plans,  and  Systems  Concepts 
(SCs)  . 

Exploratory  Development  Program  Plan  (EDPP) . The  EDPP 
is  the  authoritative  statement  of  the  content  and  rationale 
of  the  Navy's  Exploratory  Development  Program.  It  defines 
the  major  thrusts  of  the  overall  Program,  identifies  major 
Program  goals,  and  apportions  resources  among  all  elements 
of  the  Program.  Further,  it  summarizes  and  provides  an  inte- 
grated overview  of  the  Technical  Strategies  and  the  supporting 
Laboratory  and  Systems  Command  Program  Plans.  The  EDPP  is 
prepared  by  the  Director  of  Navy  Technology  (DNT)  on  the  basis 
of:  (1)  budget  and  program  guidance  from  higher  authority, 

(2)  Technical  Objectives  developed  by  the  Strategy  Team, 

(3)  Program  Plans  developed  by  Laboratories  and  SYSCOMs,  and 

(4)  Internal  and  External  Program  Review  Reports.  The  EDPP 
is  continually  updated  to  reflect  changes  in  these  inputs. 

Technical  Strategies  Technical  Strategies  serve  two 
major  purposes:  They  effectively  relate  the  Exploratory 
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Development  Program  to  Department  of  the  Navy  Mission  Needs, 
and  they  make  that  relationship  visible  and  credible. 

Functions  of  Technical  Strategies.  Technical  Strategies 
have  both  direct  and  indirect  functions. 

Direct  Functions.  Technical  Strategies  are  designed  to: 

(1)  take  maximum  advantage  of  new  technical  opportunities, 

(2)  exploit  deficiencies  in  opposition  capabilities,  and 

(3)  provide  prompt  response  to  perceived  requirements  for 
superior  naval  capability. 

Indirect  Functions.  Strategies  provide  the  vehicle  for  a 
two-way  information  flow  on  needs  and  technological  capabili- 
ties (and  possibilities) . They  also  serve  as  the  means  for 
demonstrating  the  general  rationality — and  specifically  the 
mission-relevance  in  output-oriented  terms--of  the  Exploratory 
Development  Program. 

Kinds  of  Technical  Strategies.  There  are  two  kinds  of 
Technical  Strategies:  those  which  relate  to  the  Mission 
Areas  for  which  the  Navy  has  responsibility  and  those  which 
relate  to  the  Technology  Areas  which  are  basic  to  the  Navy's 
operational  capabilities. 

Mission  Area  (MA)  Technical  Strategies.  MA  Technical 
Strategies  directly  l’nk  the  Exploratory  Development  Program 
to  the  major  missions  of  Department  of  the  Navy  operating 
forces.  MA  Strategies  are  developed  for  assigned  Mission 
Areas  or  for  established  segments  of  Mission  Areas.  Since 
MA  Strategies  are  designed  both  to  relate  the  Program  to 


91 


f 


operational  missions,  and  to  facilitate  understanding  of  that 
relationship  by  officials  and  organizations  with  influence 
over  the  Department  of  the  Navy  Program,  MA  Strategy  Areas 
are  structured  to  be  compatible  with  the  structure  of  Mission 
Areas  used  by  DDR&E  in  the  Mission  Area  Summaries  (MAS) . 

Technology  Area  (TA)  Technical  Strategies.  TA  Technical 
Strategies  set  forth  objectives  for  technology  and  the  sup- 
porting rationales.  The  structure  of  areas  for  which  TA 
Strategies  are  developed  is  also  derived  from  the  DDR&E  mission 
Area  Summaries.  TA  Technical  Strategies  so  defined  are  also 
compatible  with  the  Technology  Coordinating  Papers  (TCPs) 
issued  by  the  OSD. 

Preparation  of  Technical  Strategies . Preparation  of  both 
MA  and  TA  Technical  Strategies  involves  the  identification 
and  evaluation  of  needs,  and  the  identification  of  present, 
future,  and  potential  capabilities  for  satisfying  those  needs. 
All  Technical  Strategy  documents  employ  a similar  format. 

Each  describes  the  relevant  Strategy  Area  (this  will  be  either 
a Mission  Area  or  a Technology  Area) , sets  forth  Objectives 
and  the  supporting  rationales,  identifies  how  Objectives  in 
the  Strategy  Area  are  being  pursued  currently,  and  references 
sources  of  additional  information. 

Mission  Area  (MA)  Technical  Strategies.  In  general,  each 
MA  Technical  Strategy  sets  forth:  CD  Mission  Area  Objectives 
for  the  Mission  Area,  (2)  the  supporting  rationales  for  those 
Objectives,  and  (3)  a discussion  of  present  and  future 
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exploratory  development  efforts  relevant  to  MA  Objectives. 
Preparing  MA  Technical  Strategies  requires  a comprehensive 
understanding  of  operational  needs. 

Mission  Need  Information.  Mission  Need  information  is 
available  through  both  formal  and  informal  systems.  Informa- 
tion on  operational  needs  is  provided  primarily  through  an 
informal  face-to-face  system  based  on  and  backed  up  by  a 
document-based  system.  Information  on  needs  is  derived  from 
a variety  of  sources  including  Fleet  letters,  CNO  memoranda, 
extrapolation  of  threat  statements,  etc.  Mission  Need  infor- 
mation flows  through  the  Navy/Marine  Corps  organization  to 
the  Exploratory  Development  Program  by  means  of  three  formal 
documents:  the  Navy  Science  and  Technology  Objectives  (STOs) , 

the  Marine  Corps  General  Operational  Requirements  (GORs) , and 
the  Operational  Requirements  (ORs) . The  first  two  documents 
are  aimed  primarily  at  guiding  the  inception  of  projects  and 
programs  (although  guidance  may  be  derived  from  them  at  any 
point  in  the  process) . The  Operational  Requirements  (ORs)  are 
concerned  with  the  transition  of  System  Concepts  to  Advanced 
Development. 

Mission  Area  Objectives.  MA  Objectives  are  objectives 
for  operational  capabilities  within  the  Strategy  Areas.  They 
are  set  forth  as  means-free  statements  of  needs  for  opera- 
tional capabilities  required  to  execute  the  mission.  These 
statements  are  prepared  in  a manner  which  satisfies  the 
requirements  for  statements  of  Mission  Needs  for  the  initiation 
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of  major  system  acquisitions  as  set  forth  in  OMB  Circular 
A-109  and  implementing  DOD  directives  and  instructions.  MA 
Objectives  should  be  considered  identical  to  the  Mission 
Element  Need  Statements  (MENS)  defined  in  DOD  Directive 
5000.1  and  should  conform  to  the  format  specified  in  Appendix 
D.  All  MA  Objectives  are  to  be  numbered  in  a manner  which 
facilitates  the  information  capabilities  discussed  in  4.6. 

The  rationale  for  each  MA  Objective  should  be  explained 
clearly  and  concisely  and  should  reference  all  official  docu- 
mentation (such  as  STOs  and  GORs)  on  which  it  is  based  in 
whole  or  in  part. 

Actual  and  Potential  Capabilities.  Associated  with  each 
MA  Objective  is  a discussion  of  means  for  satisfying  the 
Objective.  This  discussion  consists  of  a concise  description 
of  each  system.  System  Concept,  or  other  way  (such  as  a combi- 
nation of  forces)  to  satisfy  the  Objective,  along  with  refer- 
ences to  primary  sources  of  additional  information.  Addition- 
ally, capabilities  relevant  to  each  Mission  Need  are  to  be 
identified  and  summarized  briefly.  These  include  present  capa- 
bilities, systems  in  development  or  production  but  not  yet 
operational,  and  Demonstrated-feasible,  Presumed-feasible, 
and  Pre-feasible  System  Concepts  (SCs) . All  SCs  will  include 
military  worth  ratings  by  CNO  and/or  CMC.  The  listing  of 
Demonstrated-feasible  SCs  will  also  identify  concepts  which 
were  dropped  since  the  last  update  of  the  Technical  Strategy. 
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Technology  Area  (TA)  Technical  Strategies,  Each  TA 
Technical  Strategy:  (1)  Describes  current  technical  efforts 
and  relates  these  to  identified  Mission  Needs,  (2)  sets  forth 
Objectives  for  technology  and  the  supporting  rationales  of 
those  Objectives,  (3)  justifies  the  importance  of  technical 
opportunities  and  potential  advancements  in  output-oriented 
terms,  and  (4)  explicitly  summarizes  and  presents  justifica- 
tions for  present,  future  and  potential  technological  capabil- 
ities by  specifically  referencing  relevant  SCs.  TA  Technical 
Strategies  are  developed  and/or  updated  in  a manner  analogous 
to  the  process  described  above  for  MA  Technical  Strategies. 

The  process  involves  defining  Technology  Needs,  setting  forth 
the  rationales  for  those  needs,  and  describing  present,  future, 
and  potential  capabilities  relevant  to  identified  Objectives. 

A Technology  Area  is  concisely  defined  in  terms  of  the  tech- 
nologies included  within  its  scope.  Where  further  clarity 
is  needed,  a Technology  Area  may  be  further  defined  by  identi- 
fying related  technologies  beyond  its  scope. 

Technology  Need  Information.  Technology  Need  information 
is  derived  from  three  principal  sources:  (1)  the  systematic 
analysis  of  Pre-feasible  SCs,  (2)  feedback  from  the  develop- 
ment and  manufacturing  processes,  and  (3)  feedback  from  the 
operating  forces.  Information  from  analysis  of  SCs  takes  the 
form  of  documented  statements  of  technological  problems  or 
barriers;  feedback  information  flows  primarily  through  person- 
to-person  links  and  is  introduced  into  the  Strategy  delibera- 
tions most  frequently  through  the  personal  experience  of 
Strategy  Team  members. 
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Technology  Area  Objectives.  Objectives  for  technological 
capabilities  are  set  forth  as  statements  of  needs  for  tech- 
nological capabilities,  termed  Technology  Area  Objectives, 
or  TA  Objectives.  These  are  statements  of  needs  both  for  the 
development  of  new  technological  capabilities  in  support  of 
SCs,  and  for  the  preservation  of  a base  of  technology  under- 
lying present  operational  capabilities  (where  it  is  judged 
that  such  a technology  base  would  be  endangered  in  the  absence 
of  action  to  preserve  it) . TA  Objectives  are  numerically 
identified  in  a manner  which  facilitates  the  information  capa- 
bilities discussed  above  and  in  4.6.  The  rationale  for  each 
TA  Objective  should  be  set  forth  clearly  and  concisely  and 
should  reference  all  official  documentation  (such  as  techno- 
logical problem  statements  from  analyses  of  SCs)  on  which  it 
is  based.  Every  rationale  should  present  a convincing  case 
which  describes  how  the  Objective  is  critical  either  to 
achievement  of  higher  level  of  mission  capability  or  to  the 
preservation  of  capabilities  which  would  be  endangered  in  the 
absence  of  actions  the  Objective  is  designed  to  stimulate. 

Actual  and  Potential  Capabilities.  Technological  capa- 
bilities relevant  to  each  TA  Objective  are  identified  and 
summarized  briefly,  as  are  the  rationales  for  each  of  those 
capabilities.  These  include  statements  and  rationales  for 
current  capabilities,  for  capabilities  anticipated  on  the 
basis  of  planned  technical  effort,  and  for  capabilities  which 
are  potentially  achievable  if  a specified  level  of  technical 
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effort  is  assumed.  The  accompanying  discussion  identifies 
the  leading  organizations  possessing,  acquiring,  or  capable 
of  acquiring  the  capabilities . Relevant  ongoing  or  planned 
technical  programs  are  identified,  and  their  predicted  impacts 
on  capabilities  are  specified. 

Program  Plans.  Program  Plans  are  the  primary  instruments 
through  which  the  DNT  exerts  top-down  influence  over  portions 
of  the  Exploratory  Development  Program  assigned  to  each  SYS- 
COM  and  Laboratory.  Program  Plans  document  each  SYSCOM's 
or  Laboratory's  integrated  and  balanced  plan  for  maximizing 
the  contribution  of  its  own  exploratory  development  efforts 
to  Department  of  the  Navy  needs.  Program  Plans  are  not 
collections  of  Task  Area  Plans  (TAPs) . (TAPs  are  prepared 
after  the  Program  Plans  are  approved  and  are  designed  to 
carry  out  objectives  listed  in  those  Plans.)  Analogous  to  a 
D CP  for  approval,  funding,  and  management  of  an  acquisition 
program,  the  Program  Plan  constitutes  a "contract"  between 
DNT  and  the  SYSCQM  03/Lab  TD  for  the  Labor atory/SYSCOM  por- 
tions of  the  6.2  Program.  The  Program  Plan  is  the  basis  for 
the  negotiation,  approval,  funding,  and  subsequent  review 
of  each  SYSCOM's  and  Laboratory's  Exploratory  Development 
Program.  The  content  and  format  for  Program  Plans  are  sum- 
marized briefly  below. 

Technology  Needs  and  Thrusts.  This  section  will  provide 
an  introduction  and  overview  of  the  proposed  technical  pro- 
gram, setting  forth  the  major  thrusts  of  the  program  and  the 
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rationale  for  those  thrusts.  It  will  also  discuss  significant 
changes  from  past  years  and  the  rationale  for  those  changes. 

Program  Objectives.  The  technological  objectives  the 
technical  program  is  designed  to  achieve  will  be  set  forth, 
as  will  the  rationale  for  tho*’  jjectives.  These  objectives 
are  to  be  stated  in  quantitativ  terms  (to  the  extent  possible) 
to  facilitate  subsequent  review  of  results.  The  rationale 
for  objectives  should  explicitly  identify  the  approved 
Technical  Objectives  which  the  program  objectives  are  designed 
to  support. 

Proposed  Program.  The  technical  program  to  achieve  each 
program  objective  will  be  briefly  described  in  both  narrative 
and  quantitative  terms.  This  description  should  include  the 
proposed  dollars  to  be  spent  in  support  of  each  program  objec- 
tive, the  prospective  performing  institution (s) , and  major 
milestones  scheduled  for  achievement  during  the  program  year. 

In-House/Con tract  Patio.  The  program  plan  will  specify 
the  relative  proportions  of  work  to  be  performed  by  Govern- 
ment organizations  (in-house)  and  by  industry  and  other  non- 
Government  organizations  (under  contract) . The  DNT  will 
negotiate  the  in-house/contract  ratios  with  each  SYSCOM  and 
Laboratory  to  ensure  that  the  overall  Program  conforms  to  the 
most  recent  policy  guidance. 

Program  Turnover.  Ongoing  efforts  should  be  critically 
reviewed  and  sufficient  work  completed  or  terminated,  to 
permit  a minimum  of  20%  of  the  technical  program  to  be  composed 
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of  new  starts.  The  Program  Plan  will  specify  the  percentage 
of  the  program  which  is  new,  and  will  identify  efforts  being 
completed  or  terminated. 

Priorities.  The  technical  program  will  be  organized  into 
decision  packages  corresponding  to  discrete  elements  of  the 
proposed  program  (such  as  all  work  related  to  specific  System 
Concept  or  Laboratory  subproject) . These  decision  packages 
will  be  presented,  along  with  cost  information,  in  a rank 
ordering.  This  is  intended  to  facilitate  the  elimination,  or 
addition,  of  entire  projects  (rather  than  to  make  across-the- 
board  funding  reductions  which  could  result  in  subcritical 
funding  and  the  inefficient  use  of  resources) . Rank  ordering 
also  facilitates  the  phased  initiation  of  new  starts  to  accom- 
modate Congressional  budget  actions  and  deferrals  of  funds. 

System  Concepts.  Priority  rankings  for  SYSCOMs  will  be 
organized  around  the  SCs  which  are  the  objectives  of  SYSCOM 
efforts.  Information  to  be  presented  will  include: 

a.  A master  rank -order  of  all  ongoing  and  new  starts  of 
all  three  classes  of  SCs. 

b.  A rank-order  of  both  ongoing  and  new  starts  for  each 
of  the  three  classes  of  SCs. 

c.  A rank-order  for  proposed  new  starts  for  each  category 
of  SC. 

Laboratory  Subprojects.  The  proposed  Laboratory  program 
will  be  presented  as  a master  rank-ordered  list  of  subprojects 
and  will  include  ongoing  work  and  proposed  new  starts. 
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System  Concepts  (SCs) . An  SC  is  a System  Concept  for 
satisfying  one  or  more  MA  Objectives.  SCs  provide  a mechanism 
for:  (1)  identifying  technology  needs,  (2)  focusing  technical 

effort  on  the  technological  advances  required  to  make  these 
high- leverage  system  concepts  feasible,  and  (3)  aiding  trans- 
fer of  that  technology  to  development  once  technical  feasi- 
bility has  been  achieved.  Although  normally  originated  by 
Laboratories,  Systems  Commands,  or  industry,  SCs  may  be  origi- 
nated by  any  organization.  Instructions  for  the  documentation 
of  SCs  are  contained  in  Appendix  E. 

Classes  of  System  Concepts.  SCs  fall  into  three  classes:  (1) 
concepts  presumed  not  yet  technologically  feasible  without 
advances  in  underlying  technology,  (2)  concepts  presumed  to 
be  technologically  feasible  within  five  years  but  for  which 
feasibility  has  yet  to  be  demonstrated,  and  (3)  concepts 
presumed  ready  for  transition  to  development.  Each  of  these 
three  classes  of  SCs  serves  a different  function  in  the  man- 
agement of  the  Exploratory  Development  Program.  All  classes 
of  SCs  provide  a basis  for  the  assessment  of  a concept's 
military  worth;  this  is  an  essential  prerequisite  to  the  fur- 
ther investment  of  technological  resources  (beyond  the  invest- 
ment required  to  define  concepts  sufficiently  to  permit 
meaningful  military  worth  assessments) . 


Pre-Feasible  System  Concepts.  Pre-feasible  SCs  provide  a 
basis  for  identifying  and  defining  technology  which  must  be 
developed  before  these  concepts  become  technologically  feasible. 
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When  communicated  to  the  technical  community  involved  in  devel- 
oping broad-based  technology,  these  technological  require- 
ments stimulate  development  of  the  requisite  technology. 
Pre-feasible  SCs  (and  their  related  requirements  for  advance- 
ments in  technology)  are  an  essential  link  in  the  relevance 
audit  trail;  they  make  it  possible  to  demonstrate  the  direct 
relevance  of  technical  effort  in  the  broad-based  technology 
category  to  operational  missions. 

Presumed-Feasible  System  Concepts.  SCs  for  which  feasi- 
bility has  been  presumed  but  not  yet  demonstrated  provide 
the  framework  for:  (1)  defining  the  requirements  for  feasi- 
bility demonstrations,  and  (2)  organizing  related  data. 
Demonstrating  the  feasibility  of  an  SC  can  be  likened  to 
assembling  a technological  building  block  kit.  As  feasibil- 
ity is  demonstrated  for  each  essential  technology,  it  is 
added  to  the  kit  until  all  essential  building  blocks  are  in 
hand.  At  that  point  the  concept  becomes  a " Demons tr a ted- 
feasible  SC"  and  is  ready  for  transfer  to  further  development. 

Demons trated-Feasible  System  Concepts.  Demons tr a ted- 
feasible  SCs  which  are  not  immediately  transferred  to  devel- 
opment may  be  dropped,  or,  they  may  be  preserved  as  options 
for  future  development  (depending  upon  the  military  worth 
assessment  of  the  system  user) . Where  a decision  has  been 
made  to  preserve  the  option  to  develop,  the  SC  provides  the 
framework  for  the  preservation  of  that  option.  The  develop- 
ment option  is  preserved  by  continually:  (1)  monitoring 


\ 

t , 


101 


relevant  technologies,  (2)  updating  the  information  package 
to  include  current  capabilities,  and  (3)  identifying  organi- 
zations which  possess  the  needed  capabilities. 

Assessment  of  the  Military  Worth  of  SCs.  The  CNO  and  CMC 
are  invited  to  assess  periodically  the  relative  military 
worths  of  all  SCs  related  to  each  MA  objective  in  their  areas 
of  warfare.  These  military  worth  ratings,  termed  "V"  ratings 
for  value,  are  to  be  included  in  the  discussion  on  MS  Objec- 
tives in  the  Technical  Strategy  documents. 

Task  Area  Plans.  Detailed  plans  for  accomplishing  the 
objectives  in  approved  Program  Plans  are  summarized  in  Task 
Area  Plans  (TAP) . Since  program  plan  objectives  are  designed 
to  support  objectives  in  approved  Technical  Strategies,  the 
Strategies  provide  amplifying  guidance  for  the  development 
of  these  more  detailed  plains. 

Information  System.  Because  of  the  direct  relationship 
between  Technical  Objectives  called  out  in  the  Technical 
Strategies  and  objectives  defined  in  the  Laboratory  and  SYS- 
COM  Program  Plans,  a relevance  audit  trail  extends  from  each 
work  unit  to  each  assigned  Department  of  the  Navy  mission. 

When  the  information  system  mandated  by  paragraph  4.b.  of 
ASN  (R&D)  memo  of  25  July  1975,  "Management  of  the  Exploratory 
Development  Program,"  is  developed  and  in  successful  operation, 
it  will  be  possible  to  make  these  relevance  relationships 
visible  through  automatic  data  processing.  An  ADP  query 
starting  with  a Mission  Need  will  be  able  to  identify  all  MA 


Objectives  called  out  in  the  MA  Technical  Strategy  for  that 
area,  all  Technology  Area  Objectives  in  support  of  MA  Objec- 
tives called  out  in  any  of  the  TA  Technical  Strategies,  all 
program  objectives  which  support  the  MA  Objectives  either 
directly  or  indirectly  through  TA  Objectives,  and  all  work 
units  supporting  these  program  objectives.  Going  the  other 
way,  ADP  capability  based  on  the  structure  of  objectives 
extending  upward  from  the  technical  program,  through  the 
Technical  Strategies  to  Department  of  the  Navy  missions,  will 
make  it  possible  to  trace  the  impact  of  each  work  unit  on 
higher-level  objectives  and  Mission  Needs. 
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STRUCTURE  OF  THE  EXPLORATORY  DEVELOPMENT  PROGRAM 

The  Chief  of  Naval  Development  (CND)  establishes  and 
promulgates  the  structural  framework  for  planning,  programming, 
and  budgeting  the  Navy's  Exploratory  Development  Program.  This 
structure  provides  a link  with  DOD  Program  Elements,  defines 
the  scope  of  work  within  specific  areas,  and  identifies  the 
organization  within  the  Navy  responsible  for  planning,  pro- 
gramming, and  budgeting. 

Building  Blocks  of  the  Exploratory  Development  Program. 

The  building  blocks  of  the  Exploratory  Development  Program 
are  described  in  the  following  paragraphs.  They  cure  listed 
in  descending  order,  from  the  largest  block  to  the  smallest. 

Program  Elements.  Program  Elements  are  the  smallest  sub- 
division of  the  R&D  program  considered  in  the  DOD  programming 
system.  A program  is  defined  as  an  integrated  activity,  an 
identifiable  military  capability;  a force,  support  activity, 
research  activity,  etc.,  comprising  a combination  of  personnel, 
equipment,  and  facilities.  In  exploratory  development,  a 
Program  Element  consists  of  a number  of  projects  in  a related 
field  within  a single  budget  activity. 

Program  Groups.  Program  groups  are  primary  divisions 
within  the  Navy  exploratory  development  planning  system.  They 
are  used  for  management  and  data-processing  purposes,  and 
provide  the  links  between  the  Navy  planning  system  and  the 
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Program  Elements  of  the  DOD  system.  They  correspond  one-to- 
one  with  the  Program  Elements— with  the  exception  of  Groups 
54  and  55,  each  of  which  corresponds  to  two  Program  Elements 
and  thus  requires  the  project  area  number  to  relate  a compo- 
nent effort  to  the  correct  Program  Element.  The  program 
groups  are  as  follows: 


Program 
Group  I 

Program  Element  Title 

Program 

Element 

11 

Undersea  Target  Surveillance 

62711N 

12 

Surface/Aerospace  Target  Surveillance 

62712N 

21 

Command  and  Control 

6 2 7 2 IN 

31 

Missile  Propulsion 

62331N 

32 

Strike  Warfare  Weaponry 

62332N 

33 

Undersea  Warfare  Weaponry 

62633N 

34 

Countermeasures 

62734N 

41 

Aircraft 

6 24 4 IN 

42 

Nuclear  Propulsion 

62542N 

43 

Ships , subs  and  boats 

62543N 

51 

Biomedical  Technology 

62758N 
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Program 
Group  ♦ 

52 

53 

54 

54 

55 

55 

56 

57 
61 


Program  Element  Title 

Ocean  and  Atmospheric  Support 
Technology 

Logistics  Technology 
Materials 

Electronic  Devices 

Training  and  Human  Engineering 

Personnel  Support  Technology 

C/B  Weapons  Defense 

Energy  and  Environmental  Protection 

Laboratory  IED 


Program 
Element  # 

52759N 

62760N 

6 2 7 6 IN 

62762N 

62757N 

62763N 

62764N 

62765N 

62766N 


Functional  Area.  The  functional  areas  are  the  five 
principal  headings  under  which  exploratory  development  is 
classified  for  program  planning  purposes.  These  functional 
areas  are: 

100  - Target  Surveillance 
200  - Command  and  Control 
300  - Weaponry 
400  - Naval  Vehicles 
500  - Support  Technology 

Project  Areas.  Project  areas  further  subdivide  the  func- 
tional areas,  as  shown  in  Appendix  and  serve  to  group 
closely-related  task  areas. 

Projects . Projects  encompass  the  aggregate  of  task  areas 
under  a given  project  area  which  fall  within  a single  program 
group  (therefore  usually  within  a given  Program  Element) . 
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Subprojects . Subprojects  encompass  the  aggregate  of  task 
areas  within  a project  which  are  assigned  to  a single  princi- 
pal development  activity.  Development  activity  identifying 


codes  are  as  follows: 

Activity  Code 

NAVAIRSYSCOM  W 

NAVELEXSYSCOM  X 

NAVFACENGCOM  Y 

NAVMEDRSCHDEVCOM  M 

ONR  R 

NAVSEASYSCOM  S 

NAVSUPSYSCOM  T 

DLP  Z 


Task  Areas.  Task  areas  encompass  exploratory  development 
effort  directed  toward  a specific  objective,  e.g.,  thrust- 
controlled  solid  rocket.  Task  areas  are  the  level  at  which 
detailed  plans  are  prepared.  These  plans  are  available  to 
the  Chief  of  Naval  Development  as  back-up  data  for  justifica- 
tion of  the  Program  to  higher  authorities,  if  needed.  A task 
area  may  consist  of  one  or  more  tasks  and  be  assigned  to  an 
individual  Laboratory,  Systems  Command,  or  Office  of 
implementation . 


Tasks.  Tasks  are  subdivisions  of  task  areas  established 
on  the  basis  of  organizations  performing  the  work.  They  are 
used  for  internal  Department  of  the  Navy  management  only. 
Specific  task  assignments  are  made  for  the  portion  of  the 
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work  under  a given  task  area  to  be  contracted  or  to  be 
assigned  to  an  individual  Laboratory  or  field  activity.  A 
task  may  consist  of  one  or  more  work  units. 

Work  Units.  Work  units  are  the  smallest  segments  into 
which  research  and  technology  efforts  are  normally  divided 
for  purposes  of  local  administration.  In  the  Exploratory 
Development  Program,  work  units  are  subdivisions  of  a task 
selected  by  the  organization  performing  the  work  to  provide 
effective  local  technical  control  and  supervision.  A work 
unit  is  technically  distinguishable  in  scope,  objective,  and/ 
or  approach  from  other  scientific  or  technical  efforts  with 
which  it  may  be  aggregated  for  financial  or  administrative 
purposes . 

Numbering  System.  Based  upon  the  building  blocks  des- 
cribed above,  the  numbering  system  applied  to  specific  devel- 
opment efforts  within  the  Exploratory  Development  Program  is 
designed  to  permit  ready  identification  and  retrieval  of  infor- 


mation regarding  any  portion  of  the  Program.  An  example  of 
the  use  of  the  system  is  shown  below: 

An  Exploratory  Development  effort  F 

in  the  "Missile  Propulsion"  Program  Group  ....  31 

in  the  "Weaponry"  Functional  Area 300 

and  specific  Project  Area  "Solid  Propulsion  ....  332 

has  Project  Number F 31  332 

The  portion  performed  by  NAVAIR  W 

has  Subproject  Number WF  31  332 
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